[==========] Running 56 tests from 4 test suites.
[----------] Global test environment set-up.
[----------] 30 tests from SecurityITest
[ RUN ] SecurityITest.TestAuthorizationOnListTablets
Loading random data
Initializing database '/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786041246094603-20232-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 18:34:06 dist-test-slave-1xrh krb5kdc[20239](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 18:34:06 dist-test-slave-1xrh krb5kdc[20239](info): set up 2 sockets
Aug 06 18:34:06 dist-test-slave-1xrh krb5kdc[20239](info): commencing operation
krb5kdc: starting...
WARNING: Logging before InitGoogleLogging() is written to STDERR
W20260806 18:34:07.198861 20232 mini_kdc.cc:121] Time spent starting KDC: real 1.068s user 0.005s sys 0.003s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786041246094603-20232-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786041246094603-20232-0/krb5kdc/test-user.keytab.
Aug 06 18:34:07 dist-test-slave-1xrh krb5kdc[20239](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041247, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T18:34:07Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T18:34:07Z Disabled control of system clock
WARNING: no policy specified for kudu/127.19.194.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.19.194.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.19.194.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786041246094603-20232-0/krb5kdc/kudu_127.19.194.62.keytab.
Entry for principal kudu/127.19.194.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786041246094603-20232-0/krb5kdc/kudu_127.19.194.62.keytab.
I20260806 18:34:07.326462 20232 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJW69tn/build/debug/bin/kudu
/tmp/dist-test-taskJW69tn/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786041246094603-20232-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786041246094603-20232-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786041246094603-20232-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786041246094603-20232-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.19.194.62:36995
--webserver_interface=127.19.194.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786041246094603-20232-0/krb5kdc/kudu_127.19.194.62.keytab
--principal=kudu/127.19.194.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.19.194.20:46719
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.19.194.62:36995
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786041246094603-20232-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786041246094603-20232-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786041246094603-20232-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 18:34:07.439450 20254 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 18:34:07.439730 20254 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 18:34:07.439782 20254 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 18:34:07.444249 20254 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 18:34:07.444299 20254 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 18:34:07.444322 20254 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 18:34:07.444339 20254 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 18:34:07.444355 20254 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 18:34:07.449321 20254 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.194.20:46719
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786041246094603-20232-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786041246094603-20232-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.19.194.62:36995
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786041246094603-20232-0/krb5kdc/kudu_127.19.194.62.keytab
--principal=kudu/127.19.194.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.19.194.62:36995
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786041246094603-20232-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.19.194.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20254
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786041246094603-20232-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 17:59:13 UTC on ccycloud.quasar-wvsnzx.root.comops.site
build id 4571028
I20260806 18:34:07.450184 20254 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 18:34:07.451052 20254 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 18:34:07.459393 20260 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:34:07.459434 20259 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 18:34:07.459434 20254 server_base.cc:1034] running on GCE node
W20260806 18:34:07.459482 20262 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 18:34:07.460044 20254 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 18:34:07.461027 20254 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 18:34:07.462188 20254 hybrid_clock.cc:648] HybridClock initialized: now 1786041247462125 us; error 74 us; skew 500 ppm
Aug 06 18:34:07 dist-test-slave-1xrh krb5kdc[20239](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041247, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.194.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 18:34:07.464919 20254 init.cc:377] Logged in from keytab as kudu/127.19.194.62@KRBTEST.COM (short username kudu)
I20260806 18:34:07.465723 20254 webserver.cc:460] Webserver started at http://127.19.194.62:33743/ using document root <none> and password file <none>
I20260806 18:34:07.466208 20254 fs_manager.cc:359] Metadata directory not provided
I20260806 18:34:07.466262 20254 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 18:34:07.466440 20254 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 18:34:07.468429 20254 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786041246094603-20232-0/minicluster-data/master-0/data/instance:
uuid: "118c8146171f4156a440a3362698a2b3"
format_stamp: "Formatted at 2026-08-06 18:34:07 on dist-test-slave-1xrh"
server_key: "78a1ed8f961ff061ed68d2f3b04b8e7b"
server_key_iv: "5621969299e9e4ca8bf3d09eafb601b8"
server_key_version: "encryptionkey@0"
I20260806 18:34:07.468945 20254 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786041246094603-20232-0/minicluster-data/master-0/wal/instance:
uuid: "118c8146171f4156a440a3362698a2b3"
format_stamp: "Formatted at 2026-08-06 18:34:07 on dist-test-slave-1xrh"
server_key: "78a1ed8f961ff061ed68d2f3b04b8e7b"
server_key_iv: "5621969299e9e4ca8bf3d09eafb601b8"
server_key_version: "encryptionkey@0"
I20260806 18:34:07.472440 20254 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20260806 18:34:07.474593 20269 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 18:34:07.475757 20254 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260806 18:34:07.475891 20254 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786041246094603-20232-0/minicluster-data/master-0/data,/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786041246094603-20232-0/minicluster-data/master-0/wal
uuid: "118c8146171f4156a440a3362698a2b3"
format_stamp: "Formatted at 2026-08-06 18:34:07 on dist-test-slave-1xrh"
server_key: "78a1ed8f961ff061ed68d2f3b04b8e7b"
server_key_iv: "5621969299e9e4ca8bf3d09eafb601b8"
server_key_version: "encryptionkey@0"
I20260806 18:34:07.475986 20254 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786041246094603-20232-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786041246094603-20232-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786041246094603-20232-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 18:34:07.489290 20254 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 18:34:07.492112 20254 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 18:34:07.492300 20254 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 18:34:07.499402 20254 rpc_server.cc:307] RPC server started. Bound to: 127.19.194.62:36995
I20260806 18:34:07.499429 20322 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.194.62:36995 every 8 connection(s)
I20260806 18:34:07.500288 20254 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786041246094603-20232-0/minicluster-data/master-0/data/info.pb
I20260806 18:34:07.502554 20232 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJW69tn/build/debug/bin/kudu as pid 20254
I20260806 18:34:07.502672 20232 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786041246094603-20232-0/minicluster-data/master-0/wal/instance
I20260806 18:34:07.503006 20323 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 18:34:07.504022 20232 external_mini_cluster.cc:1428] Setting key 528bc7a5bc35da4bc742f8d99a61a451
I20260806 18:34:07.510113 20323 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 118c8146171f4156a440a3362698a2b3: Bootstrap starting.
I20260806 18:34:07.512753 20323 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 118c8146171f4156a440a3362698a2b3: Neither blocks nor log segments found. Creating new log.
I20260806 18:34:07.514051 20323 log.cc:824] T 00000000000000000000000000000000 P 118c8146171f4156a440a3362698a2b3: Log is configured to *not* fsync() on all Append() calls
I20260806 18:34:07.516016 20323 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 118c8146171f4156a440a3362698a2b3: No bootstrap required, opened a new log
I20260806 18:34:07.520171 20323 raft_consensus.cc:348] T 00000000000000000000000000000000 P 118c8146171f4156a440a3362698a2b3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "118c8146171f4156a440a3362698a2b3" member_type: VOTER last_known_addr { host: "127.19.194.62" port: 36995 } }
I20260806 18:34:07.520395 20323 raft_consensus.cc:374] T 00000000000000000000000000000000 P 118c8146171f4156a440a3362698a2b3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 18:34:07.520489 20323 raft_consensus.cc:729] T 00000000000000000000000000000000 P 118c8146171f4156a440a3362698a2b3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 118c8146171f4156a440a3362698a2b3, State: Initialized, Role: FOLLOWER
I20260806 18:34:07.520960 20323 consensus_queue.cc:260] T 00000000000000000000000000000000 P 118c8146171f4156a440a3362698a2b3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "118c8146171f4156a440a3362698a2b3" member_type: VOTER last_known_addr { host: "127.19.194.62" port: 36995 } }
I20260806 18:34:07.521136 20323 raft_consensus.cc:388] T 00000000000000000000000000000000 P 118c8146171f4156a440a3362698a2b3 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 18:34:07.521214 20323 raft_consensus.cc:482] T 00000000000000000000000000000000 P 118c8146171f4156a440a3362698a2b3 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 18:34:07.521317 20323 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 118c8146171f4156a440a3362698a2b3 [term 0 FOLLOWER]: Advancing to term 1
I20260806 18:34:07.522586 20323 raft_consensus.cc:504] T 00000000000000000000000000000000 P 118c8146171f4156a440a3362698a2b3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "118c8146171f4156a440a3362698a2b3" member_type: VOTER last_known_addr { host: "127.19.194.62" port: 36995 } }
I20260806 18:34:07.522939 20323 leader_election.cc:302] T 00000000000000000000000000000000 P 118c8146171f4156a440a3362698a2b3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 118c8146171f4156a440a3362698a2b3; no voters:
Aug 06 18:34:07 dist-test-slave-1xrh krb5kdc[20239](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041247, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.19.194.62@KRBTEST.COM
I20260806 18:34:07.523283 20323 leader_election.cc:288] T 00000000000000000000000000000000 P 118c8146171f4156a440a3362698a2b3 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 18:34:07.523384 20328 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 118c8146171f4156a440a3362698a2b3 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 18:34:07.523702 20328 raft_consensus.cc:686] T 00000000000000000000000000000000 P 118c8146171f4156a440a3362698a2b3 [term 1 LEADER]: Becoming Leader. State: Replica: 118c8146171f4156a440a3362698a2b3, State: Running, Role: LEADER
I20260806 18:34:07.524044 20328 consensus_queue.cc:237] T 00000000000000000000000000000000 P 118c8146171f4156a440a3362698a2b3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "118c8146171f4156a440a3362698a2b3" member_type: VOTER last_known_addr { host: "127.19.194.62" port: 36995 } }
I20260806 18:34:07.524717 20323 sys_catalog.cc:564] T 00000000000000000000000000000000 P 118c8146171f4156a440a3362698a2b3 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 18:34:07.525238 20328 sys_catalog.cc:455] T 00000000000000000000000000000000 P 118c8146171f4156a440a3362698a2b3 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 118c8146171f4156a440a3362698a2b3. Latest consensus state: current_term: 1 leader_uuid: "118c8146171f4156a440a3362698a2b3" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "118c8146171f4156a440a3362698a2b3" member_type: VOTER last_known_addr { host: "127.19.194.62" port: 36995 } } }
I20260806 18:34:07.525362 20328 sys_catalog.cc:458] T 00000000000000000000000000000000 P 118c8146171f4156a440a3362698a2b3 [sys.catalog]: This master's current role is: LEADER
I20260806 18:34:07.526060 20333 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 18:34:07.529037 20329 sys_catalog.cc:455] T 00000000000000000000000000000000 P 118c8146171f4156a440a3362698a2b3 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "118c8146171f4156a440a3362698a2b3" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "118c8146171f4156a440a3362698a2b3" member_type: VOTER last_known_addr { host: "127.19.194.62" port: 36995 } } }
I20260806 18:34:07.529170 20329 sys_catalog.cc:458] T 00000000000000000000000000000000 P 118c8146171f4156a440a3362698a2b3 [sys.catalog]: This master's current role is: LEADER
I20260806 18:34:07.530781 20325 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:34:07.507237 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51854 (local address 127.19.194.62:36995)
0806 18:34:07.507679 (+ 442us) server_negotiation.cc:206] Beginning negotiation
0806 18:34:07.507704 (+ 25us) server_negotiation.cc:399] Waiting for connection header
0806 18:34:07.508715 (+ 1011us) server_negotiation.cc:407] Connection header received
0806 18:34:07.512983 (+ 4268us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:34:07.513000 (+ 17us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:34:07.513311 (+ 311us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:34:07.513634 (+ 323us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:34:07.516371 (+ 2737us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:34:07.517295 (+ 924us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:34:07.518538 (+ 1243us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:34:07.518976 (+ 438us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:34:07.519192 (+ 216us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:34:07.524310 (+ 5118us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:34:07.524317 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:34:07.524332 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 18:34:07.524362 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:34:07.526571 (+ 2209us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:34:07.527285 (+ 714us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:34:07.527292 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:34:07.527298 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:34:07.527423 (+ 125us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:34:07.527803 (+ 380us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:34:07.527806 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:34:07.527807 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:34:07.528249 (+ 442us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:34:07.529387 (+ 1138us) server_negotiation.cc:1035] Waiting for connection context
0806 18:34:07.529592 (+ 205us) server_negotiation.cc:299] Negotiation successful
0806 18:34:07.529892 (+ 300us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":289,"thread_start_us":155,"threads_started":1}
I20260806 18:34:07.532262 20333 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 18:34:07.538035 20333 catalog_manager.cc:1347] Generated new cluster ID: 18309e3fe12e4e00808a3b3d94c07cea
I20260806 18:34:07.538148 20333 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 18:34:07.552536 20333 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 18:34:07.553529 20333 catalog_manager.cc:1504] Loading token signing keys...
I20260806 18:34:07.563107 20333 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 118c8146171f4156a440a3362698a2b3: Generated new TSK 0
I20260806 18:34:07.563863 20333 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.19.194.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.19.194.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.19.194.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786041246094603-20232-0/krb5kdc/kudu_127.19.194.1.keytab.
Entry for principal kudu/127.19.194.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786041246094603-20232-0/krb5kdc/kudu_127.19.194.1.keytab.
I20260806 18:34:07.599704 20232 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJW69tn/build/debug/bin/kudu
/tmp/dist-test-taskJW69tn/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786041246094603-20232-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786041246094603-20232-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786041246094603-20232-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786041246094603-20232-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.19.194.1:0
--local_ip_for_outbound_sockets=127.19.194.1
--webserver_interface=127.19.194.1
--webserver_port=0
--tserver_master_addrs=127.19.194.62:36995
--keytab_file=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786041246094603-20232-0/krb5kdc/kudu_127.19.194.1.keytab
--principal=kudu/127.19.194.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.19.194.20:46719
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786041246094603-20232-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786041246094603-20232-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786041246094603-20232-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 18:34:07.710384 20349 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 18:34:07.710620 20349 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 18:34:07.710662 20349 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 18:34:07.714473 20349 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 18:34:07.714520 20349 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 18:34:07.714582 20349 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.19.194.1
I20260806 18:34:07.719688 20349 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.194.20:46719
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786041246094603-20232-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786041246094603-20232-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786041246094603-20232-0/krb5kdc/kudu_127.19.194.1.keytab
--principal=kudu/127.19.194.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.19.194.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786041246094603-20232-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.19.194.1
--webserver_port=0
--tserver_master_addrs=127.19.194.62:36995
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20349
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.19.194.1
--log_dir=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786041246094603-20232-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 17:59:13 UTC on ccycloud.quasar-wvsnzx.root.comops.site
build id 4571028
I20260806 18:34:07.720553 20349 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 18:34:07.721437 20349 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 18:34:07.731882 20354 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:34:07.731930 20357 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:34:07.731901 20355 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 18:34:07.732064 20349 server_base.cc:1034] running on GCE node
I20260806 18:34:07.732548 20349 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 18:34:07.733170 20349 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 18:34:07.734354 20349 hybrid_clock.cc:648] HybridClock initialized: now 1786041247734329 us; error 37 us; skew 500 ppm
Aug 06 18:34:07 dist-test-slave-1xrh krb5kdc[20239](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041247, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.194.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 18:34:07.737018 20349 init.cc:377] Logged in from keytab as kudu/127.19.194.1@KRBTEST.COM (short username kudu)
I20260806 18:34:07.737802 20349 webserver.cc:460] Webserver started at http://127.19.194.1:38835/ using document root <none> and password file <none>
I20260806 18:34:07.738341 20349 fs_manager.cc:359] Metadata directory not provided
I20260806 18:34:07.738435 20349 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 18:34:07.738665 20349 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 18:34:07.740700 20349 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786041246094603-20232-0/minicluster-data/ts-0/data/instance:
uuid: "fa370065655a4a13b731b3b92e201fee"
format_stamp: "Formatted at 2026-08-06 18:34:07 on dist-test-slave-1xrh"
server_key: "5385f454f303ec0a416dd605126a55aa"
server_key_iv: "23b1b10259f1c1f41405028466a3f139"
server_key_version: "encryptionkey@0"
I20260806 18:34:07.741264 20349 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786041246094603-20232-0/minicluster-data/ts-0/wal/instance:
uuid: "fa370065655a4a13b731b3b92e201fee"
format_stamp: "Formatted at 2026-08-06 18:34:07 on dist-test-slave-1xrh"
server_key: "5385f454f303ec0a416dd605126a55aa"
server_key_iv: "23b1b10259f1c1f41405028466a3f139"
server_key_version: "encryptionkey@0"
I20260806 18:34:07.745157 20349 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260806 18:34:07.747449 20364 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 18:34:07.748577 20349 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260806 18:34:07.748710 20349 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786041246094603-20232-0/minicluster-data/ts-0/data,/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786041246094603-20232-0/minicluster-data/ts-0/wal
uuid: "fa370065655a4a13b731b3b92e201fee"
format_stamp: "Formatted at 2026-08-06 18:34:07 on dist-test-slave-1xrh"
server_key: "5385f454f303ec0a416dd605126a55aa"
server_key_iv: "23b1b10259f1c1f41405028466a3f139"
server_key_version: "encryptionkey@0"
I20260806 18:34:07.748818 20349 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786041246094603-20232-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786041246094603-20232-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786041246094603-20232-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 18:34:07.764065 20349 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 18:34:07.766676 20349 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 18:34:07.766865 20349 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 18:34:07.767506 20349 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 18:34:07.768286 20349 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 18:34:07.768356 20349 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 18:34:07.768421 20349 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 18:34:07.768476 20349 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 18:34:07.777396 20349 rpc_server.cc:307] RPC server started. Bound to: 127.19.194.1:42007
I20260806 18:34:07.777422 20483 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.194.1:42007 every 8 connection(s)
I20260806 18:34:07.778194 20349 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786041246094603-20232-0/minicluster-data/ts-0/data/info.pb
I20260806 18:34:07.785936 20232 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJW69tn/build/debug/bin/kudu as pid 20349
I20260806 18:34:07.786060 20232 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786041246094603-20232-0/minicluster-data/ts-0/wal/instance
I20260806 18:34:07.786350 20232 external_mini_cluster.cc:1428] Setting key 79afde7ed929c6206b47fc2f38407f80
Aug 06 18:34:07 dist-test-slave-1xrh krb5kdc[20239](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041247, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.194.1@KRBTEST.COM for kudu/127.19.194.62@KRBTEST.COM
I20260806 18:34:07.791265 20325 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:34:07.779962 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.194.1:50837 (local address 127.19.194.62:36995)
0806 18:34:07.780117 (+ 155us) server_negotiation.cc:206] Beginning negotiation
0806 18:34:07.780122 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 18:34:07.781116 (+ 994us) server_negotiation.cc:407] Connection header received
0806 18:34:07.781911 (+ 795us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:34:07.781914 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:34:07.781961 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:34:07.782050 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:34:07.783194 (+ 1144us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:34:07.783735 (+ 541us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:34:07.784797 (+ 1062us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:34:07.785020 (+ 223us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:34:07.785094 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:34:07.787734 (+ 2640us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:34:07.787738 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:34:07.787740 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 18:34:07.787764 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:34:07.789203 (+ 1439us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:34:07.789725 (+ 522us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:34:07.789734 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:34:07.789736 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:34:07.789813 (+ 77us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:34:07.790162 (+ 349us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:34:07.790167 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:34:07.790169 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:34:07.790347 (+ 178us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:34:07.790453 (+ 106us) server_negotiation.cc:1035] Waiting for connection context
0806 18:34:07.790927 (+ 474us) server_negotiation.cc:299] Negotiation successful
0806 18:34:07.791061 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260806 18:34:07.791961 20486 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:34:07.780332 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.194.62:36995 (local address 127.19.194.1:50837)
0806 18:34:07.780979 (+ 647us) negotiation.cc:107] Waiting for socket to connect
0806 18:34:07.781018 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0806 18:34:07.781743 (+ 725us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:34:07.782214 (+ 471us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:34:07.782221 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:34:07.782565 (+ 344us) client_negotiation.cc:189] Negotiated authn=SASL
0806 18:34:07.783022 (+ 457us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:34:07.783038 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:34:07.783849 (+ 811us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:34:07.783851 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:34:07.784694 (+ 843us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:34:07.784703 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:34:07.785143 (+ 440us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:34:07.785146 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:34:07.785278 (+ 132us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:34:07.785800 (+ 522us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 18:34:07.785824 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0806 18:34:07.787565 (+ 1741us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 18:34:07.789330 (+ 1765us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:34:07.789336 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:34:07.789350 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:34:07.789609 (+ 259us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:34:07.789905 (+ 296us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:34:07.789909 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:34:07.789911 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:34:07.790063 (+ 152us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:34:07.790455 (+ 392us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 18:34:07.790471 (+ 16us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 18:34:07.790687 (+ 216us) client_negotiation.cc:769] Sending connection context
0806 18:34:07.790865 (+ 178us) client_negotiation.cc:240] Negotiation successful
0806 18:34:07.791196 (+ 331us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":465,"thread_start_us":117,"threads_started":1}
I20260806 18:34:07.793183 20484 heartbeater.cc:347] Connected to a master server at 127.19.194.62:36995
I20260806 18:34:07.793447 20484 heartbeater.cc:464] Registering TS with master...
I20260806 18:34:07.794034 20484 heartbeater.cc:511] Master 127.19.194.62:36995 requested a full tablet report, sending...
I20260806 18:34:07.795781 20286 ts_manager.cc:194] Registered new tserver with Master: fa370065655a4a13b731b3b92e201fee (127.19.194.1:42007)
WARNING: no policy specified for kudu/127.19.194.2@KRBTEST.COM; defaulting to no policy
I20260806 18:34:07.797165 20286 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.19.194.1@KRBTEST.COM'} at 127.19.194.1:50837
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.19.194.2@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.19.194.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786041246094603-20232-0/krb5kdc/kudu_127.19.194.2.keytab.
Entry for principal kudu/127.19.194.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786041246094603-20232-0/krb5kdc/kudu_127.19.194.2.keytab.
I20260806 18:34:07.815029 20232 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJW69tn/build/debug/bin/kudu
/tmp/dist-test-taskJW69tn/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786041246094603-20232-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786041246094603-20232-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786041246094603-20232-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786041246094603-20232-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.19.194.2:0
--local_ip_for_outbound_sockets=127.19.194.2
--webserver_interface=127.19.194.2
--webserver_port=0
--tserver_master_addrs=127.19.194.62:36995
--keytab_file=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786041246094603-20232-0/krb5kdc/kudu_127.19.194.2.keytab
--principal=kudu/127.19.194.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.19.194.20:46719
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786041246094603-20232-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786041246094603-20232-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786041246094603-20232-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 18:34:07.922849 20489 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 18:34:07.923077 20489 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 18:34:07.923116 20489 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 18:34:07.927268 20489 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 18:34:07.927317 20489 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 18:34:07.927415 20489 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.19.194.2
I20260806 18:34:07.932583 20489 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.194.20:46719
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786041246094603-20232-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786041246094603-20232-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786041246094603-20232-0/krb5kdc/kudu_127.19.194.2.keytab
--principal=kudu/127.19.194.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.19.194.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786041246094603-20232-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.19.194.2
--webserver_port=0
--tserver_master_addrs=127.19.194.62:36995
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20489
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.19.194.2
--log_dir=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786041246094603-20232-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 17:59:13 UTC on ccycloud.quasar-wvsnzx.root.comops.site
build id 4571028
I20260806 18:34:07.933491 20489 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 18:34:07.934407 20489 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 18:34:07.943369 20494 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:34:07.943341 20497 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 18:34:07.943347 20489 server_base.cc:1034] running on GCE node
W20260806 18:34:07.943394 20495 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 18:34:07.943934 20489 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 18:34:07.944514 20489 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 18:34:07.945672 20489 hybrid_clock.cc:648] HybridClock initialized: now 1786041247945646 us; error 36 us; skew 500 ppm
Aug 06 18:34:07 dist-test-slave-1xrh krb5kdc[20239](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041247, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.194.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 18:34:07.948300 20489 init.cc:377] Logged in from keytab as kudu/127.19.194.2@KRBTEST.COM (short username kudu)
I20260806 18:34:07.949081 20489 webserver.cc:460] Webserver started at http://127.19.194.2:37029/ using document root <none> and password file <none>
I20260806 18:34:07.949616 20489 fs_manager.cc:359] Metadata directory not provided
I20260806 18:34:07.949674 20489 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 18:34:07.949892 20489 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 18:34:07.951927 20489 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786041246094603-20232-0/minicluster-data/ts-1/data/instance:
uuid: "567c7a9306f74b009370a0ee1cb66c54"
format_stamp: "Formatted at 2026-08-06 18:34:07 on dist-test-slave-1xrh"
server_key: "8ac76d2aa273f3dc2ebbbb2347cb91d9"
server_key_iv: "dbfda60d923d8911e980f5d4b886607c"
server_key_version: "encryptionkey@0"
I20260806 18:34:07.952546 20489 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786041246094603-20232-0/minicluster-data/ts-1/wal/instance:
uuid: "567c7a9306f74b009370a0ee1cb66c54"
format_stamp: "Formatted at 2026-08-06 18:34:07 on dist-test-slave-1xrh"
server_key: "8ac76d2aa273f3dc2ebbbb2347cb91d9"
server_key_iv: "dbfda60d923d8911e980f5d4b886607c"
server_key_version: "encryptionkey@0"
I20260806 18:34:07.956167 20489 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.001s sys 0.004s
I20260806 18:34:07.958458 20504 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 18:34:07.959698 20489 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260806 18:34:07.959831 20489 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786041246094603-20232-0/minicluster-data/ts-1/data,/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786041246094603-20232-0/minicluster-data/ts-1/wal
uuid: "567c7a9306f74b009370a0ee1cb66c54"
format_stamp: "Formatted at 2026-08-06 18:34:07 on dist-test-slave-1xrh"
server_key: "8ac76d2aa273f3dc2ebbbb2347cb91d9"
server_key_iv: "dbfda60d923d8911e980f5d4b886607c"
server_key_version: "encryptionkey@0"
I20260806 18:34:07.959935 20489 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786041246094603-20232-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786041246094603-20232-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786041246094603-20232-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 18:34:07.970662 20489 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 18:34:07.973738 20489 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 18:34:07.973937 20489 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 18:34:07.974470 20489 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 18:34:07.975263 20489 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 18:34:07.975309 20489 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 18:34:07.975376 20489 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 18:34:07.975411 20489 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 18:34:07.983565 20489 rpc_server.cc:307] RPC server started. Bound to: 127.19.194.2:37417
I20260806 18:34:07.983587 20625 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.194.2:37417 every 8 connection(s)
I20260806 18:34:07.984408 20489 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786041246094603-20232-0/minicluster-data/ts-1/data/info.pb
I20260806 18:34:07.990499 20232 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJW69tn/build/debug/bin/kudu as pid 20489
I20260806 18:34:07.990605 20232 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786041246094603-20232-0/minicluster-data/ts-1/wal/instance
I20260806 18:34:07.990865 20232 external_mini_cluster.cc:1428] Setting key a0ed47008859d9f6049191096de1bbf3
Aug 06 18:34:07 dist-test-slave-1xrh krb5kdc[20239](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041247, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.194.2@KRBTEST.COM for kudu/127.19.194.62@KRBTEST.COM
I20260806 18:34:07.997524 20325 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:34:07.986160 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.194.2:34815 (local address 127.19.194.62:36995)
0806 18:34:07.986311 (+ 151us) server_negotiation.cc:206] Beginning negotiation
0806 18:34:07.986316 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 18:34:07.987326 (+ 1010us) server_negotiation.cc:407] Connection header received
0806 18:34:07.988105 (+ 779us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:34:07.988107 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:34:07.988151 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:34:07.988222 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:34:07.989268 (+ 1046us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:34:07.989775 (+ 507us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:34:07.991242 (+ 1467us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:34:07.991487 (+ 245us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:34:07.991566 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:34:07.994063 (+ 2497us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:34:07.994072 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:34:07.994074 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 18:34:07.994097 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:34:07.995546 (+ 1449us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:34:07.996036 (+ 490us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:34:07.996040 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:34:07.996041 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:34:07.996119 (+ 78us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:34:07.996448 (+ 329us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:34:07.996452 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:34:07.996454 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:34:07.996622 (+ 168us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:34:07.996772 (+ 150us) server_negotiation.cc:1035] Waiting for connection context
0806 18:34:07.997245 (+ 473us) server_negotiation.cc:299] Negotiation successful
0806 18:34:07.997359 (+ 114us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260806 18:34:07.998273 20628 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:34:07.986523 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.194.62:36995 (local address 127.19.194.2:34815)
0806 18:34:07.987197 (+ 674us) negotiation.cc:107] Waiting for socket to connect
0806 18:34:07.987237 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0806 18:34:07.987929 (+ 692us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:34:07.988376 (+ 447us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:34:07.988382 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:34:07.988702 (+ 320us) client_negotiation.cc:189] Negotiated authn=SASL
0806 18:34:07.989145 (+ 443us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:34:07.989157 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:34:07.989877 (+ 720us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:34:07.989880 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:34:07.991074 (+ 1194us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:34:07.991086 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:34:07.991619 (+ 533us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:34:07.991624 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:34:07.991822 (+ 198us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:34:07.992462 (+ 640us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 18:34:07.992489 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0806 18:34:07.993902 (+ 1413us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 18:34:07.995663 (+ 1761us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:34:07.995668 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:34:07.995681 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:34:07.995933 (+ 252us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:34:07.996209 (+ 276us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:34:07.996212 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:34:07.996214 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:34:07.996348 (+ 134us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:34:07.996772 (+ 424us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 18:34:07.996780 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 18:34:07.997015 (+ 235us) client_negotiation.cc:769] Sending connection context
0806 18:34:07.997199 (+ 184us) client_negotiation.cc:240] Negotiation successful
0806 18:34:07.997509 (+ 310us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":444,"thread_start_us":107,"threads_started":1}
I20260806 18:34:07.999672 20626 heartbeater.cc:347] Connected to a master server at 127.19.194.62:36995
I20260806 18:34:07.999938 20626 heartbeater.cc:464] Registering TS with master...
WARNING: no policy specified for kudu/127.19.194.3@KRBTEST.COM; defaulting to no policy
I20260806 18:34:08.000411 20626 heartbeater.cc:511] Master 127.19.194.62:36995 requested a full tablet report, sending...
I20260806 18:34:08.001462 20286 ts_manager.cc:194] Registered new tserver with Master: 567c7a9306f74b009370a0ee1cb66c54 (127.19.194.2:37417)
I20260806 18:34:08.002058 20286 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.19.194.2@KRBTEST.COM'} at 127.19.194.2:34815
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.19.194.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.19.194.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786041246094603-20232-0/krb5kdc/kudu_127.19.194.3.keytab.
Entry for principal kudu/127.19.194.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786041246094603-20232-0/krb5kdc/kudu_127.19.194.3.keytab.
I20260806 18:34:08.017858 20232 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJW69tn/build/debug/bin/kudu
/tmp/dist-test-taskJW69tn/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786041246094603-20232-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786041246094603-20232-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786041246094603-20232-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786041246094603-20232-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.19.194.3:0
--local_ip_for_outbound_sockets=127.19.194.3
--webserver_interface=127.19.194.3
--webserver_port=0
--tserver_master_addrs=127.19.194.62:36995
--keytab_file=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786041246094603-20232-0/krb5kdc/kudu_127.19.194.3.keytab
--principal=kudu/127.19.194.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.19.194.20:46719
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786041246094603-20232-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786041246094603-20232-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786041246094603-20232-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 18:34:08.128834 20631 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 18:34:08.129113 20631 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 18:34:08.129187 20631 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 18:34:08.133306 20631 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 18:34:08.133381 20631 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 18:34:08.133473 20631 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.19.194.3
I20260806 18:34:08.138669 20631 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.194.20:46719
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786041246094603-20232-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786041246094603-20232-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786041246094603-20232-0/krb5kdc/kudu_127.19.194.3.keytab
--principal=kudu/127.19.194.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.19.194.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786041246094603-20232-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.19.194.3
--webserver_port=0
--tserver_master_addrs=127.19.194.62:36995
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20631
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.19.194.3
--log_dir=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786041246094603-20232-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 17:59:13 UTC on ccycloud.quasar-wvsnzx.root.comops.site
build id 4571028
I20260806 18:34:08.139752 20631 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 18:34:08.140745 20631 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 18:34:08.150141 20637 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:34:08.150215 20639 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:34:08.150134 20636 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 18:34:08.150319 20631 server_base.cc:1034] running on GCE node
I20260806 18:34:08.150797 20631 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 18:34:08.151475 20631 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 18:34:08.152664 20631 hybrid_clock.cc:648] HybridClock initialized: now 1786041248152657 us; error 29 us; skew 500 ppm
Aug 06 18:34:08 dist-test-slave-1xrh krb5kdc[20239](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041248, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.194.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 18:34:08.155577 20631 init.cc:377] Logged in from keytab as kudu/127.19.194.3@KRBTEST.COM (short username kudu)
I20260806 18:34:08.156394 20631 webserver.cc:460] Webserver started at http://127.19.194.3:42031/ using document root <none> and password file <none>
I20260806 18:34:08.156929 20631 fs_manager.cc:359] Metadata directory not provided
I20260806 18:34:08.157019 20631 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 18:34:08.157244 20631 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 18:34:08.159215 20631 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786041246094603-20232-0/minicluster-data/ts-2/data/instance:
uuid: "77f3841243bd4248a201105d4328604f"
format_stamp: "Formatted at 2026-08-06 18:34:08 on dist-test-slave-1xrh"
server_key: "04c9a243573990d215cae46954e8c8bb"
server_key_iv: "640d8a0667532d7dc1897790fe5bbeb8"
server_key_version: "encryptionkey@0"
I20260806 18:34:08.159783 20631 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786041246094603-20232-0/minicluster-data/ts-2/wal/instance:
uuid: "77f3841243bd4248a201105d4328604f"
format_stamp: "Formatted at 2026-08-06 18:34:08 on dist-test-slave-1xrh"
server_key: "04c9a243573990d215cae46954e8c8bb"
server_key_iv: "640d8a0667532d7dc1897790fe5bbeb8"
server_key_version: "encryptionkey@0"
I20260806 18:34:08.163405 20631 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.002s sys 0.002s
I20260806 18:34:08.165834 20646 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 18:34:08.167022 20631 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260806 18:34:08.167171 20631 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786041246094603-20232-0/minicluster-data/ts-2/data,/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786041246094603-20232-0/minicluster-data/ts-2/wal
uuid: "77f3841243bd4248a201105d4328604f"
format_stamp: "Formatted at 2026-08-06 18:34:08 on dist-test-slave-1xrh"
server_key: "04c9a243573990d215cae46954e8c8bb"
server_key_iv: "640d8a0667532d7dc1897790fe5bbeb8"
server_key_version: "encryptionkey@0"
I20260806 18:34:08.167294 20631 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786041246094603-20232-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786041246094603-20232-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786041246094603-20232-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 18:34:08.187108 20631 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 18:34:08.189934 20631 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 18:34:08.190167 20631 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 18:34:08.190742 20631 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 18:34:08.191665 20631 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 18:34:08.191747 20631 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 18:34:08.191818 20631 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 18:34:08.191856 20631 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 18:34:08.200763 20631 rpc_server.cc:307] RPC server started. Bound to: 127.19.194.3:38201
I20260806 18:34:08.200793 20766 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.194.3:38201 every 8 connection(s)
I20260806 18:34:08.201608 20631 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786041246094603-20232-0/minicluster-data/ts-2/data/info.pb
I20260806 18:34:08.203876 20232 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJW69tn/build/debug/bin/kudu as pid 20631
I20260806 18:34:08.203996 20232 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786041246094603-20232-0/minicluster-data/ts-2/wal/instance
I20260806 18:34:08.204245 20232 external_mini_cluster.cc:1428] Setting key 2ee388697d13baf83fe0ce437ec2e291
Aug 06 18:34:08 dist-test-slave-1xrh krb5kdc[20239](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041248, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.194.3@KRBTEST.COM for kudu/127.19.194.62@KRBTEST.COM
I20260806 18:34:08.214735 20325 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:34:08.203339 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.194.3:48721 (local address 127.19.194.62:36995)
0806 18:34:08.203479 (+ 140us) server_negotiation.cc:206] Beginning negotiation
0806 18:34:08.203483 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 18:34:08.204619 (+ 1136us) server_negotiation.cc:407] Connection header received
0806 18:34:08.205706 (+ 1087us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:34:08.205710 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:34:08.205768 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:34:08.205873 (+ 105us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:34:08.207008 (+ 1135us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:34:08.207602 (+ 594us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:34:08.208741 (+ 1139us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:34:08.208966 (+ 225us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:34:08.209096 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:34:08.211435 (+ 2339us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:34:08.211438 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:34:08.211440 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 18:34:08.211465 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:34:08.212651 (+ 1186us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:34:08.213179 (+ 528us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:34:08.213183 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:34:08.213185 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:34:08.213267 (+ 82us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:34:08.213658 (+ 391us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:34:08.213663 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:34:08.213664 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:34:08.213849 (+ 185us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:34:08.213958 (+ 109us) server_negotiation.cc:1035] Waiting for connection context
0806 18:34:08.214434 (+ 476us) server_negotiation.cc:299] Negotiation successful
0806 18:34:08.214553 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260806 18:34:08.215600 20769 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:34:08.203717 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.194.62:36995 (local address 127.19.194.3:48721)
0806 18:34:08.204441 (+ 724us) negotiation.cc:107] Waiting for socket to connect
0806 18:34:08.204492 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0806 18:34:08.205501 (+ 1009us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:34:08.206050 (+ 549us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:34:08.206057 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:34:08.206404 (+ 347us) client_negotiation.cc:189] Negotiated authn=SASL
0806 18:34:08.206862 (+ 458us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:34:08.206875 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:34:08.207741 (+ 866us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:34:08.207744 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:34:08.208608 (+ 864us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:34:08.208618 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:34:08.209134 (+ 516us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:34:08.209138 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:34:08.209317 (+ 179us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:34:08.209834 (+ 517us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 18:34:08.209850 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0806 18:34:08.211269 (+ 1419us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 18:34:08.212775 (+ 1506us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:34:08.212781 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:34:08.212794 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:34:08.213041 (+ 247us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:34:08.213362 (+ 321us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:34:08.213365 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:34:08.213367 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:34:08.213557 (+ 190us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:34:08.213972 (+ 415us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 18:34:08.213980 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 18:34:08.214197 (+ 217us) client_negotiation.cc:769] Sending connection context
0806 18:34:08.214377 (+ 180us) client_negotiation.cc:240] Negotiation successful
0806 18:34:08.214699 (+ 322us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":526,"thread_start_us":143,"threads_started":1}
I20260806 18:34:08.216645 20767 heartbeater.cc:347] Connected to a master server at 127.19.194.62:36995
I20260806 18:34:08.216867 20767 heartbeater.cc:464] Registering TS with master...
I20260806 18:34:08.217329 20767 heartbeater.cc:511] Master 127.19.194.62:36995 requested a full tablet report, sending...
I20260806 18:34:08.218413 20285 ts_manager.cc:194] Registered new tserver with Master: 77f3841243bd4248a201105d4328604f (127.19.194.3:38201)
I20260806 18:34:08.218997 20285 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.19.194.3@KRBTEST.COM'} at 127.19.194.3:48721
I20260806 18:34:08.230011 20232 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 06 18:34:08 dist-test-slave-1xrh krb5kdc[20239](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041248, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 06 18:34:08 dist-test-slave-1xrh krb5kdc[20239](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041248, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.19.194.1@KRBTEST.COM
I20260806 18:34:08.256888 20774 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:34:08.247118 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33494 (local address 127.19.194.1:42007)
0806 18:34:08.247476 (+ 358us) server_negotiation.cc:206] Beginning negotiation
0806 18:34:08.247489 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0806 18:34:08.247514 (+ 25us) server_negotiation.cc:407] Connection header received
0806 18:34:08.247602 (+ 88us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:34:08.247608 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:34:08.247790 (+ 182us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:34:08.247908 (+ 118us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:34:08.248627 (+ 719us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:34:08.249687 (+ 1060us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:34:08.250490 (+ 803us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:34:08.250845 (+ 355us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:34:08.250929 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:34:08.253673 (+ 2744us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:34:08.253682 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:34:08.253694 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 18:34:08.253718 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:34:08.255269 (+ 1551us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:34:08.255651 (+ 382us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:34:08.255656 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:34:08.255657 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:34:08.255734 (+ 77us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:34:08.255957 (+ 223us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:34:08.255960 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:34:08.255961 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:34:08.256197 (+ 236us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:34:08.256364 (+ 167us) server_negotiation.cc:1035] Waiting for connection context
0806 18:34:08.256511 (+ 147us) server_negotiation.cc:299] Negotiation successful
0806 18:34:08.256683 (+ 172us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":255,"thread_start_us":131,"threads_started":1}
W20260806 18:34:08.257856 20394 server_base.cc:1116] Unauthorized access attempt to method kudu.tserver.TabletServerService.ListTablets from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:33494
Aug 06 18:34:08 dist-test-slave-1xrh krb5kdc[20239](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041248, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 06 18:34:08 dist-test-slave-1xrh krb5kdc[20239](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041248, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.19.194.1@KRBTEST.COM
I20260806 18:34:08.282768 20774 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:34:08.275094 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33502 (local address 127.19.194.1:42007)
0806 18:34:08.275305 (+ 211us) server_negotiation.cc:206] Beginning negotiation
0806 18:34:08.275308 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 18:34:08.275362 (+ 54us) server_negotiation.cc:407] Connection header received
0806 18:34:08.275471 (+ 109us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:34:08.275474 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:34:08.275513 (+ 39us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:34:08.275584 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:34:08.276210 (+ 626us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:34:08.276719 (+ 509us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:34:08.277532 (+ 813us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:34:08.277745 (+ 213us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:34:08.277830 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:34:08.280132 (+ 2302us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:34:08.280135 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:34:08.280137 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 18:34:08.280159 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:34:08.281408 (+ 1249us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:34:08.281802 (+ 394us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:34:08.281805 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:34:08.281806 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:34:08.281877 (+ 71us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:34:08.282105 (+ 228us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:34:08.282108 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:34:08.282109 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:34:08.282259 (+ 150us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:34:08.282353 (+ 94us) server_negotiation.cc:1035] Waiting for connection context
0806 18:34:08.282558 (+ 205us) server_negotiation.cc:299] Negotiation successful
0806 18:34:08.282652 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":103}
I20260806 18:34:08.283847 20232 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJW69tn/build/debug/bin/kudu with pid 20349
I20260806 18:34:08.291242 20232 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJW69tn/build/debug/bin/kudu with pid 20489
I20260806 18:34:08.298382 20232 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJW69tn/build/debug/bin/kudu with pid 20631
I20260806 18:34:08.304426 20232 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJW69tn/build/debug/bin/kudu with pid 20254
2026-08-06T18:34:08Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizationOnListTablets (2205 ms)
[ RUN ] SecurityITest.TestAuthorizationOnChecksum
Loading random data
Initializing database '/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786041246094603-20232-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 18:34:08 dist-test-slave-1xrh krb5kdc[20781](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 18:34:08 dist-test-slave-1xrh krb5kdc[20781](info): set up 2 sockets
Aug 06 18:34:08 dist-test-slave-1xrh krb5kdc[20781](info): commencing operation
krb5kdc: starting...
W20260806 18:34:09.356248 20232 mini_kdc.cc:121] Time spent starting KDC: real 1.028s user 0.003s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786041246094603-20232-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786041246094603-20232-0/krb5kdc/test-user.keytab.
Aug 06 18:34:09 dist-test-slave-1xrh krb5kdc[20781](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041249, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T18:34:09Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T18:34:09Z Disabled control of system clock
WARNING: no policy specified for kudu/127.19.194.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.19.194.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.19.194.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786041246094603-20232-0/krb5kdc/kudu_127.19.194.62.keytab.
Entry for principal kudu/127.19.194.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786041246094603-20232-0/krb5kdc/kudu_127.19.194.62.keytab.
I20260806 18:34:09.471997 20232 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJW69tn/build/debug/bin/kudu
/tmp/dist-test-taskJW69tn/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786041246094603-20232-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786041246094603-20232-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786041246094603-20232-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786041246094603-20232-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.19.194.62:37651
--webserver_interface=127.19.194.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786041246094603-20232-0/krb5kdc/kudu_127.19.194.62.keytab
--principal=kudu/127.19.194.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.19.194.20:45071
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.19.194.62:37651
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786041246094603-20232-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786041246094603-20232-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786041246094603-20232-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 18:34:09.578972 20796 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 18:34:09.579278 20796 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 18:34:09.579366 20796 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 18:34:09.583202 20796 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 18:34:09.583278 20796 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 18:34:09.583323 20796 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 18:34:09.583377 20796 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 18:34:09.583411 20796 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 18:34:09.589223 20796 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.194.20:45071
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786041246094603-20232-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786041246094603-20232-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.19.194.62:37651
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786041246094603-20232-0/krb5kdc/kudu_127.19.194.62.keytab
--principal=kudu/127.19.194.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.19.194.62:37651
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786041246094603-20232-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.19.194.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20796
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786041246094603-20232-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 17:59:13 UTC on ccycloud.quasar-wvsnzx.root.comops.site
build id 4571028
I20260806 18:34:09.590530 20796 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 18:34:09.591578 20796 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 18:34:09.599949 20805 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:34:09.600056 20803 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:34:09.599925 20802 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 18:34:09.600291 20796 server_base.cc:1034] running on GCE node
I20260806 18:34:09.600833 20796 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 18:34:09.601881 20796 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 18:34:09.603067 20796 hybrid_clock.cc:648] HybridClock initialized: now 1786041249603041 us; error 40 us; skew 500 ppm
Aug 06 18:34:09 dist-test-slave-1xrh krb5kdc[20781](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041249, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.194.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 18:34:09.605672 20796 init.cc:377] Logged in from keytab as kudu/127.19.194.62@KRBTEST.COM (short username kudu)
I20260806 18:34:09.606532 20796 webserver.cc:460] Webserver started at http://127.19.194.62:40719/ using document root <none> and password file <none>
I20260806 18:34:09.607108 20796 fs_manager.cc:359] Metadata directory not provided
I20260806 18:34:09.607241 20796 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 18:34:09.607494 20796 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 18:34:09.609434 20796 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786041246094603-20232-0/minicluster-data/master-0/data/instance:
uuid: "8bc5d948b58041678c23ef3086b07c76"
format_stamp: "Formatted at 2026-08-06 18:34:09 on dist-test-slave-1xrh"
server_key: "6b63798c62d7893d339a31c041500a31"
server_key_iv: "73ba7241f7f8848bd542148f01818a72"
server_key_version: "encryptionkey@0"
I20260806 18:34:09.609983 20796 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786041246094603-20232-0/minicluster-data/master-0/wal/instance:
uuid: "8bc5d948b58041678c23ef3086b07c76"
format_stamp: "Formatted at 2026-08-06 18:34:09 on dist-test-slave-1xrh"
server_key: "6b63798c62d7893d339a31c041500a31"
server_key_iv: "73ba7241f7f8848bd542148f01818a72"
server_key_version: "encryptionkey@0"
I20260806 18:34:09.613615 20796 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.000s sys 0.005s
I20260806 18:34:09.615849 20812 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 18:34:09.617084 20796 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.001s
I20260806 18:34:09.617249 20796 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786041246094603-20232-0/minicluster-data/master-0/data,/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786041246094603-20232-0/minicluster-data/master-0/wal
uuid: "8bc5d948b58041678c23ef3086b07c76"
format_stamp: "Formatted at 2026-08-06 18:34:09 on dist-test-slave-1xrh"
server_key: "6b63798c62d7893d339a31c041500a31"
server_key_iv: "73ba7241f7f8848bd542148f01818a72"
server_key_version: "encryptionkey@0"
I20260806 18:34:09.617349 20796 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786041246094603-20232-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786041246094603-20232-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786041246094603-20232-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 18:34:09.634071 20796 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 18:34:09.636767 20796 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 18:34:09.636951 20796 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 18:34:09.644096 20796 rpc_server.cc:307] RPC server started. Bound to: 127.19.194.62:37651
I20260806 18:34:09.644115 20865 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.194.62:37651 every 8 connection(s)
I20260806 18:34:09.644970 20796 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786041246094603-20232-0/minicluster-data/master-0/data/info.pb
I20260806 18:34:09.647469 20232 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJW69tn/build/debug/bin/kudu as pid 20796
I20260806 18:34:09.647586 20232 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786041246094603-20232-0/minicluster-data/master-0/wal/instance
I20260806 18:34:09.647622 20866 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 18:34:09.647863 20232 external_mini_cluster.cc:1428] Setting key 414953a648fda31719b01bea6b7a201b
I20260806 18:34:09.654623 20866 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8bc5d948b58041678c23ef3086b07c76: Bootstrap starting.
Aug 06 18:34:09 dist-test-slave-1xrh krb5kdc[20781](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041249, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.19.194.62@KRBTEST.COM
I20260806 18:34:09.657174 20866 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 8bc5d948b58041678c23ef3086b07c76: Neither blocks nor log segments found. Creating new log.
I20260806 18:34:09.658084 20866 log.cc:824] T 00000000000000000000000000000000 P 8bc5d948b58041678c23ef3086b07c76: Log is configured to *not* fsync() on all Append() calls
I20260806 18:34:09.660409 20866 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8bc5d948b58041678c23ef3086b07c76: No bootstrap required, opened a new log
I20260806 18:34:09.662887 20869 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:34:09.649219 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40320 (local address 127.19.194.62:37651)
0806 18:34:09.649659 (+ 440us) server_negotiation.cc:206] Beginning negotiation
0806 18:34:09.649678 (+ 19us) server_negotiation.cc:399] Waiting for connection header
0806 18:34:09.649721 (+ 43us) server_negotiation.cc:407] Connection header received
0806 18:34:09.650380 (+ 659us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:34:09.650402 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:34:09.650826 (+ 424us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:34:09.651216 (+ 390us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:34:09.651867 (+ 651us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:34:09.653383 (+ 1516us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:34:09.654194 (+ 811us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:34:09.654869 (+ 675us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:34:09.655030 (+ 161us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:34:09.657759 (+ 2729us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:34:09.657764 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:34:09.657776 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 18:34:09.657803 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:34:09.659919 (+ 2116us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:34:09.660330 (+ 411us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:34:09.660336 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:34:09.660341 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:34:09.660498 (+ 157us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:34:09.660783 (+ 285us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:34:09.660786 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:34:09.660788 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:34:09.661251 (+ 463us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:34:09.661439 (+ 188us) server_negotiation.cc:1035] Waiting for connection context
0806 18:34:09.661627 (+ 188us) server_negotiation.cc:299] Negotiation successful
0806 18:34:09.661906 (+ 279us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":278,"thread_start_us":116,"threads_started":1}
I20260806 18:34:09.664796 20866 raft_consensus.cc:348] T 00000000000000000000000000000000 P 8bc5d948b58041678c23ef3086b07c76 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8bc5d948b58041678c23ef3086b07c76" member_type: VOTER last_known_addr { host: "127.19.194.62" port: 37651 } }
I20260806 18:34:09.665069 20866 raft_consensus.cc:374] T 00000000000000000000000000000000 P 8bc5d948b58041678c23ef3086b07c76 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 18:34:09.665143 20866 raft_consensus.cc:729] T 00000000000000000000000000000000 P 8bc5d948b58041678c23ef3086b07c76 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8bc5d948b58041678c23ef3086b07c76, State: Initialized, Role: FOLLOWER
I20260806 18:34:09.665553 20866 consensus_queue.cc:260] T 00000000000000000000000000000000 P 8bc5d948b58041678c23ef3086b07c76 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8bc5d948b58041678c23ef3086b07c76" member_type: VOTER last_known_addr { host: "127.19.194.62" port: 37651 } }
I20260806 18:34:09.665696 20866 raft_consensus.cc:388] T 00000000000000000000000000000000 P 8bc5d948b58041678c23ef3086b07c76 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 18:34:09.665760 20866 raft_consensus.cc:482] T 00000000000000000000000000000000 P 8bc5d948b58041678c23ef3086b07c76 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 18:34:09.665853 20866 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 8bc5d948b58041678c23ef3086b07c76 [term 0 FOLLOWER]: Advancing to term 1
I20260806 18:34:09.666669 20866 raft_consensus.cc:504] T 00000000000000000000000000000000 P 8bc5d948b58041678c23ef3086b07c76 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8bc5d948b58041678c23ef3086b07c76" member_type: VOTER last_known_addr { host: "127.19.194.62" port: 37651 } }
I20260806 18:34:09.666998 20866 leader_election.cc:302] T 00000000000000000000000000000000 P 8bc5d948b58041678c23ef3086b07c76 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 8bc5d948b58041678c23ef3086b07c76; no voters:
I20260806 18:34:09.667312 20866 leader_election.cc:288] T 00000000000000000000000000000000 P 8bc5d948b58041678c23ef3086b07c76 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 18:34:09.667423 20871 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 8bc5d948b58041678c23ef3086b07c76 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 18:34:09.667687 20871 raft_consensus.cc:686] T 00000000000000000000000000000000 P 8bc5d948b58041678c23ef3086b07c76 [term 1 LEADER]: Becoming Leader. State: Replica: 8bc5d948b58041678c23ef3086b07c76, State: Running, Role: LEADER
I20260806 18:34:09.667999 20871 consensus_queue.cc:237] T 00000000000000000000000000000000 P 8bc5d948b58041678c23ef3086b07c76 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8bc5d948b58041678c23ef3086b07c76" member_type: VOTER last_known_addr { host: "127.19.194.62" port: 37651 } }
I20260806 18:34:09.668496 20866 sys_catalog.cc:564] T 00000000000000000000000000000000 P 8bc5d948b58041678c23ef3086b07c76 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 18:34:09.669562 20873 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8bc5d948b58041678c23ef3086b07c76 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 8bc5d948b58041678c23ef3086b07c76. Latest consensus state: current_term: 1 leader_uuid: "8bc5d948b58041678c23ef3086b07c76" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8bc5d948b58041678c23ef3086b07c76" member_type: VOTER last_known_addr { host: "127.19.194.62" port: 37651 } } }
I20260806 18:34:09.669709 20873 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8bc5d948b58041678c23ef3086b07c76 [sys.catalog]: This master's current role is: LEADER
I20260806 18:34:09.670043 20872 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8bc5d948b58041678c23ef3086b07c76 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "8bc5d948b58041678c23ef3086b07c76" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8bc5d948b58041678c23ef3086b07c76" member_type: VOTER last_known_addr { host: "127.19.194.62" port: 37651 } } }
I20260806 18:34:09.670137 20872 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8bc5d948b58041678c23ef3086b07c76 [sys.catalog]: This master's current role is: LEADER
I20260806 18:34:09.670208 20880 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 18:34:09.673157 20880 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 18:34:09.678304 20880 catalog_manager.cc:1347] Generated new cluster ID: 4ddd136ea33742b0bca877a3489727f0
I20260806 18:34:09.678364 20880 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 18:34:09.693320 20880 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 18:34:09.694180 20880 catalog_manager.cc:1504] Loading token signing keys...
I20260806 18:34:09.698529 20880 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 8bc5d948b58041678c23ef3086b07c76: Generated new TSK 0
I20260806 18:34:09.699100 20880 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.19.194.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.19.194.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.19.194.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786041246094603-20232-0/krb5kdc/kudu_127.19.194.1.keytab.
Entry for principal kudu/127.19.194.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786041246094603-20232-0/krb5kdc/kudu_127.19.194.1.keytab.
I20260806 18:34:09.732232 20232 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJW69tn/build/debug/bin/kudu
/tmp/dist-test-taskJW69tn/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786041246094603-20232-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786041246094603-20232-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786041246094603-20232-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786041246094603-20232-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.19.194.1:0
--local_ip_for_outbound_sockets=127.19.194.1
--webserver_interface=127.19.194.1
--webserver_port=0
--tserver_master_addrs=127.19.194.62:37651
--keytab_file=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786041246094603-20232-0/krb5kdc/kudu_127.19.194.1.keytab
--principal=kudu/127.19.194.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.19.194.20:45071
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786041246094603-20232-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786041246094603-20232-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786041246094603-20232-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 18:34:09.858382 20892 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 18:34:09.858726 20892 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 18:34:09.858786 20892 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 18:34:09.862579 20892 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 18:34:09.862632 20892 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 18:34:09.862686 20892 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.19.194.1
I20260806 18:34:09.867651 20892 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.194.20:45071
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786041246094603-20232-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786041246094603-20232-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786041246094603-20232-0/krb5kdc/kudu_127.19.194.1.keytab
--principal=kudu/127.19.194.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.19.194.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786041246094603-20232-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.19.194.1
--webserver_port=0
--tserver_master_addrs=127.19.194.62:37651
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20892
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.19.194.1
--log_dir=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786041246094603-20232-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 17:59:13 UTC on ccycloud.quasar-wvsnzx.root.comops.site
build id 4571028
I20260806 18:34:09.868613 20892 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 18:34:09.869683 20892 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 18:34:09.878983 20899 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:34:09.879052 20901 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:34:09.878988 20898 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 18:34:09.879168 20892 server_base.cc:1034] running on GCE node
I20260806 18:34:09.879614 20892 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 18:34:09.880251 20892 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 18:34:09.881428 20892 hybrid_clock.cc:648] HybridClock initialized: now 1786041249881413 us; error 25 us; skew 500 ppm
Aug 06 18:34:09 dist-test-slave-1xrh krb5kdc[20781](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041249, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.194.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 18:34:09.884215 20892 init.cc:377] Logged in from keytab as kudu/127.19.194.1@KRBTEST.COM (short username kudu)
I20260806 18:34:09.884994 20892 webserver.cc:460] Webserver started at http://127.19.194.1:46409/ using document root <none> and password file <none>
I20260806 18:34:09.885514 20892 fs_manager.cc:359] Metadata directory not provided
I20260806 18:34:09.885576 20892 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 18:34:09.885728 20892 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 18:34:09.887671 20892 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786041246094603-20232-0/minicluster-data/ts-0/data/instance:
uuid: "ab02a1498b104224b37835f5e62baeb3"
format_stamp: "Formatted at 2026-08-06 18:34:09 on dist-test-slave-1xrh"
server_key: "3f613d2227557b1842365f7ad479eaed"
server_key_iv: "ae417424b28d50842c217f114a045551"
server_key_version: "encryptionkey@0"
I20260806 18:34:09.888245 20892 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786041246094603-20232-0/minicluster-data/ts-0/wal/instance:
uuid: "ab02a1498b104224b37835f5e62baeb3"
format_stamp: "Formatted at 2026-08-06 18:34:09 on dist-test-slave-1xrh"
server_key: "3f613d2227557b1842365f7ad479eaed"
server_key_iv: "ae417424b28d50842c217f114a045551"
server_key_version: "encryptionkey@0"
I20260806 18:34:09.891988 20892 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20260806 18:34:09.894156 20908 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 18:34:09.895299 20892 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260806 18:34:09.895386 20892 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786041246094603-20232-0/minicluster-data/ts-0/data,/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786041246094603-20232-0/minicluster-data/ts-0/wal
uuid: "ab02a1498b104224b37835f5e62baeb3"
format_stamp: "Formatted at 2026-08-06 18:34:09 on dist-test-slave-1xrh"
server_key: "3f613d2227557b1842365f7ad479eaed"
server_key_iv: "ae417424b28d50842c217f114a045551"
server_key_version: "encryptionkey@0"
I20260806 18:34:09.895462 20892 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786041246094603-20232-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786041246094603-20232-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786041246094603-20232-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 18:34:09.904774 20892 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 18:34:09.907657 20892 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 18:34:09.907793 20892 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 18:34:09.908250 20892 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 18:34:09.908982 20892 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 18:34:09.909055 20892 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 18:34:09.909127 20892 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 18:34:09.909171 20892 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 18:34:09.917837 20892 rpc_server.cc:307] RPC server started. Bound to: 127.19.194.1:39137
I20260806 18:34:09.917884 21025 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.194.1:39137 every 8 connection(s)
I20260806 18:34:09.918710 20892 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786041246094603-20232-0/minicluster-data/ts-0/data/info.pb
Aug 06 18:34:09 dist-test-slave-1xrh krb5kdc[20781](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041249, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.194.1@KRBTEST.COM for kudu/127.19.194.62@KRBTEST.COM
I20260806 18:34:09.928716 20232 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJW69tn/build/debug/bin/kudu as pid 20892
I20260806 18:34:09.928843 20232 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786041246094603-20232-0/minicluster-data/ts-0/wal/instance
I20260806 18:34:09.929093 20232 external_mini_cluster.cc:1428] Setting key 154b17080d7f5132681c7550fe53c0c7
I20260806 18:34:09.931387 20869 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:34:09.920417 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.194.1:36533 (local address 127.19.194.62:37651)
0806 18:34:09.920551 (+ 134us) server_negotiation.cc:206] Beginning negotiation
0806 18:34:09.920555 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 18:34:09.921385 (+ 830us) server_negotiation.cc:407] Connection header received
0806 18:34:09.922213 (+ 828us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:34:09.922216 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:34:09.922265 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:34:09.922339 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:34:09.923478 (+ 1139us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:34:09.924035 (+ 557us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:34:09.925089 (+ 1054us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:34:09.925330 (+ 241us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:34:09.925406 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:34:09.927645 (+ 2239us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:34:09.927648 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:34:09.927651 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 18:34:09.927675 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:34:09.929250 (+ 1575us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:34:09.929776 (+ 526us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:34:09.929779 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:34:09.929781 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:34:09.929864 (+ 83us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:34:09.930200 (+ 336us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:34:09.930203 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:34:09.930205 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:34:09.930384 (+ 179us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:34:09.930499 (+ 115us) server_negotiation.cc:1035] Waiting for connection context
0806 18:34:09.930949 (+ 450us) server_negotiation.cc:299] Negotiation successful
0806 18:34:09.931098 (+ 149us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260806 18:34:09.931960 21028 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:34:09.920780 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.194.62:37651 (local address 127.19.194.1:36533)
0806 18:34:09.921229 (+ 449us) negotiation.cc:107] Waiting for socket to connect
0806 18:34:09.921290 (+ 61us) client_negotiation.cc:174] Beginning negotiation
0806 18:34:09.922031 (+ 741us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:34:09.922497 (+ 466us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:34:09.922503 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:34:09.922851 (+ 348us) client_negotiation.cc:189] Negotiated authn=SASL
0806 18:34:09.923342 (+ 491us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:34:09.923357 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:34:09.924137 (+ 780us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:34:09.924140 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:34:09.924985 (+ 845us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:34:09.924994 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:34:09.925454 (+ 460us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:34:09.925456 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:34:09.925604 (+ 148us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:34:09.926077 (+ 473us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 18:34:09.926091 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0806 18:34:09.927491 (+ 1400us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 18:34:09.929362 (+ 1871us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:34:09.929367 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:34:09.929380 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:34:09.929661 (+ 281us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:34:09.929954 (+ 293us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:34:09.929958 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:34:09.929960 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:34:09.930092 (+ 132us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:34:09.930496 (+ 404us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 18:34:09.930502 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 18:34:09.930713 (+ 211us) client_negotiation.cc:769] Sending connection context
0806 18:34:09.930862 (+ 149us) client_negotiation.cc:240] Negotiation successful
0806 18:34:09.931096 (+ 234us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":246,"thread_start_us":102,"threads_started":1}
I20260806 18:34:09.933283 21026 heartbeater.cc:347] Connected to a master server at 127.19.194.62:37651
I20260806 18:34:09.933517 21026 heartbeater.cc:464] Registering TS with master...
I20260806 18:34:09.933970 21026 heartbeater.cc:511] Master 127.19.194.62:37651 requested a full tablet report, sending...
I20260806 18:34:09.935472 20829 ts_manager.cc:194] Registered new tserver with Master: ab02a1498b104224b37835f5e62baeb3 (127.19.194.1:39137)
I20260806 18:34:09.936630 20829 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.19.194.1@KRBTEST.COM'} at 127.19.194.1:36533
WARNING: no policy specified for kudu/127.19.194.2@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.19.194.2@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.19.194.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786041246094603-20232-0/krb5kdc/kudu_127.19.194.2.keytab.
Entry for principal kudu/127.19.194.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786041246094603-20232-0/krb5kdc/kudu_127.19.194.2.keytab.
I20260806 18:34:09.959023 20232 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJW69tn/build/debug/bin/kudu
/tmp/dist-test-taskJW69tn/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786041246094603-20232-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786041246094603-20232-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786041246094603-20232-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786041246094603-20232-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.19.194.2:0
--local_ip_for_outbound_sockets=127.19.194.2
--webserver_interface=127.19.194.2
--webserver_port=0
--tserver_master_addrs=127.19.194.62:37651
--keytab_file=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786041246094603-20232-0/krb5kdc/kudu_127.19.194.2.keytab
--principal=kudu/127.19.194.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.19.194.20:45071
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786041246094603-20232-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786041246094603-20232-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786041246094603-20232-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 18:34:10.069283 21031 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 18:34:10.069579 21031 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 18:34:10.069665 21031 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 18:34:10.073678 21031 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 18:34:10.073732 21031 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 18:34:10.073839 21031 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.19.194.2
I20260806 18:34:10.078927 21031 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.194.20:45071
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786041246094603-20232-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786041246094603-20232-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786041246094603-20232-0/krb5kdc/kudu_127.19.194.2.keytab
--principal=kudu/127.19.194.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.19.194.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786041246094603-20232-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.19.194.2
--webserver_port=0
--tserver_master_addrs=127.19.194.62:37651
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21031
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.19.194.2
--log_dir=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786041246094603-20232-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 17:59:13 UTC on ccycloud.quasar-wvsnzx.root.comops.site
build id 4571028
I20260806 18:34:10.079882 21031 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 18:34:10.080824 21031 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 18:34:10.090122 21039 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:34:10.090127 21037 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:34:10.090122 21036 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 18:34:10.090370 21031 server_base.cc:1034] running on GCE node
I20260806 18:34:10.090757 21031 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 18:34:10.091416 21031 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 18:34:10.092595 21031 hybrid_clock.cc:648] HybridClock initialized: now 1786041250092579 us; error 28 us; skew 500 ppm
Aug 06 18:34:10 dist-test-slave-1xrh krb5kdc[20781](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041250, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.194.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 18:34:10.095346 21031 init.cc:377] Logged in from keytab as kudu/127.19.194.2@KRBTEST.COM (short username kudu)
I20260806 18:34:10.096202 21031 webserver.cc:460] Webserver started at http://127.19.194.2:38453/ using document root <none> and password file <none>
I20260806 18:34:10.096742 21031 fs_manager.cc:359] Metadata directory not provided
I20260806 18:34:10.096834 21031 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 18:34:10.097054 21031 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 18:34:10.099265 21031 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786041246094603-20232-0/minicluster-data/ts-1/data/instance:
uuid: "d5642a5cae6f42b3b2aa3a8159c7820f"
format_stamp: "Formatted at 2026-08-06 18:34:10 on dist-test-slave-1xrh"
server_key: "7591508a05b472c1547225d88a6b600a"
server_key_iv: "43c14898ce5346a930d021fd7848abbb"
server_key_version: "encryptionkey@0"
I20260806 18:34:10.099835 21031 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786041246094603-20232-0/minicluster-data/ts-1/wal/instance:
uuid: "d5642a5cae6f42b3b2aa3a8159c7820f"
format_stamp: "Formatted at 2026-08-06 18:34:10 on dist-test-slave-1xrh"
server_key: "7591508a05b472c1547225d88a6b600a"
server_key_iv: "43c14898ce5346a930d021fd7848abbb"
server_key_version: "encryptionkey@0"
I20260806 18:34:10.104311 21031 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260806 18:34:10.106743 21046 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 18:34:10.107897 21031 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260806 18:34:10.108040 21031 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786041246094603-20232-0/minicluster-data/ts-1/data,/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786041246094603-20232-0/minicluster-data/ts-1/wal
uuid: "d5642a5cae6f42b3b2aa3a8159c7820f"
format_stamp: "Formatted at 2026-08-06 18:34:10 on dist-test-slave-1xrh"
server_key: "7591508a05b472c1547225d88a6b600a"
server_key_iv: "43c14898ce5346a930d021fd7848abbb"
server_key_version: "encryptionkey@0"
I20260806 18:34:10.108160 21031 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786041246094603-20232-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786041246094603-20232-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786041246094603-20232-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 18:34:10.123287 21031 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 18:34:10.126276 21031 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 18:34:10.126458 21031 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 18:34:10.127002 21031 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 18:34:10.127817 21031 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 18:34:10.127885 21031 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 18:34:10.127951 21031 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 18:34:10.127998 21031 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 18:34:10.136943 21031 rpc_server.cc:307] RPC server started. Bound to: 127.19.194.2:39009
I20260806 18:34:10.137027 21166 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.194.2:39009 every 8 connection(s)
I20260806 18:34:10.137816 21031 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786041246094603-20232-0/minicluster-data/ts-1/data/info.pb
I20260806 18:34:10.145171 20232 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJW69tn/build/debug/bin/kudu as pid 21031
I20260806 18:34:10.145258 20232 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786041246094603-20232-0/minicluster-data/ts-1/wal/instance
I20260806 18:34:10.145538 20232 external_mini_cluster.cc:1428] Setting key 5fbb7aa02f9e58eb7e580ff2a0414a20
Aug 06 18:34:10 dist-test-slave-1xrh krb5kdc[20781](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041250, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.194.2@KRBTEST.COM for kudu/127.19.194.62@KRBTEST.COM
I20260806 18:34:10.150411 20869 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:34:10.139698 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.194.2:56525 (local address 127.19.194.62:37651)
0806 18:34:10.139824 (+ 126us) server_negotiation.cc:206] Beginning negotiation
0806 18:34:10.139828 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 18:34:10.140622 (+ 794us) server_negotiation.cc:407] Connection header received
0806 18:34:10.141422 (+ 800us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:34:10.141424 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:34:10.141479 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:34:10.141553 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:34:10.142582 (+ 1029us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:34:10.143102 (+ 520us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:34:10.144225 (+ 1123us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:34:10.144449 (+ 224us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:34:10.144531 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:34:10.146937 (+ 2406us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:34:10.146940 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:34:10.146942 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 18:34:10.146964 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:34:10.148439 (+ 1475us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:34:10.148954 (+ 515us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:34:10.148959 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:34:10.148960 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:34:10.149043 (+ 83us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:34:10.149382 (+ 339us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:34:10.149386 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:34:10.149388 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:34:10.149563 (+ 175us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:34:10.149665 (+ 102us) server_negotiation.cc:1035] Waiting for connection context
0806 18:34:10.150122 (+ 457us) server_negotiation.cc:299] Negotiation successful
0806 18:34:10.150238 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260806 18:34:10.151108 21169 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:34:10.140043 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.194.62:37651 (local address 127.19.194.2:56525)
0806 18:34:10.140480 (+ 437us) negotiation.cc:107] Waiting for socket to connect
0806 18:34:10.140518 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0806 18:34:10.141237 (+ 719us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:34:10.141705 (+ 468us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:34:10.141711 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:34:10.142027 (+ 316us) client_negotiation.cc:189] Negotiated authn=SASL
0806 18:34:10.142445 (+ 418us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:34:10.142456 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:34:10.143254 (+ 798us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:34:10.143256 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:34:10.144116 (+ 860us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:34:10.144125 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:34:10.144573 (+ 448us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:34:10.144576 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:34:10.144707 (+ 131us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:34:10.145264 (+ 557us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 18:34:10.145287 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0806 18:34:10.146732 (+ 1445us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 18:34:10.148569 (+ 1837us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:34:10.148575 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:34:10.148587 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:34:10.148850 (+ 263us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:34:10.149135 (+ 285us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:34:10.149138 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:34:10.149140 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:34:10.149288 (+ 148us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:34:10.149665 (+ 377us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 18:34:10.149673 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 18:34:10.149899 (+ 226us) client_negotiation.cc:769] Sending connection context
0806 18:34:10.150074 (+ 175us) client_negotiation.cc:240] Negotiation successful
0806 18:34:10.150358 (+ 284us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":265,"thread_start_us":114,"threads_started":1}
I20260806 18:34:10.152371 21167 heartbeater.cc:347] Connected to a master server at 127.19.194.62:37651
I20260806 18:34:10.152591 21167 heartbeater.cc:464] Registering TS with master...
I20260806 18:34:10.153043 21167 heartbeater.cc:511] Master 127.19.194.62:37651 requested a full tablet report, sending...
I20260806 18:34:10.154135 20829 ts_manager.cc:194] Registered new tserver with Master: d5642a5cae6f42b3b2aa3a8159c7820f (127.19.194.2:39009)
I20260806 18:34:10.154995 20829 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.19.194.2@KRBTEST.COM'} at 127.19.194.2:56525
WARNING: no policy specified for kudu/127.19.194.3@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.19.194.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.19.194.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786041246094603-20232-0/krb5kdc/kudu_127.19.194.3.keytab.
Entry for principal kudu/127.19.194.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786041246094603-20232-0/krb5kdc/kudu_127.19.194.3.keytab.
I20260806 18:34:10.172351 20232 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJW69tn/build/debug/bin/kudu
/tmp/dist-test-taskJW69tn/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786041246094603-20232-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786041246094603-20232-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786041246094603-20232-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786041246094603-20232-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.19.194.3:0
--local_ip_for_outbound_sockets=127.19.194.3
--webserver_interface=127.19.194.3
--webserver_port=0
--tserver_master_addrs=127.19.194.62:37651
--keytab_file=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786041246094603-20232-0/krb5kdc/kudu_127.19.194.3.keytab
--principal=kudu/127.19.194.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.19.194.20:45071
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786041246094603-20232-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786041246094603-20232-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786041246094603-20232-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 18:34:10.280529 21172 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 18:34:10.280741 21172 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 18:34:10.280794 21172 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 18:34:10.284555 21172 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 18:34:10.284600 21172 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 18:34:10.284655 21172 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.19.194.3
I20260806 18:34:10.289703 21172 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.194.20:45071
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786041246094603-20232-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786041246094603-20232-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786041246094603-20232-0/krb5kdc/kudu_127.19.194.3.keytab
--principal=kudu/127.19.194.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.19.194.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786041246094603-20232-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.19.194.3
--webserver_port=0
--tserver_master_addrs=127.19.194.62:37651
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21172
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.19.194.3
--log_dir=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786041246094603-20232-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 17:59:13 UTC on ccycloud.quasar-wvsnzx.root.comops.site
build id 4571028
I20260806 18:34:10.290520 21172 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 18:34:10.291415 21172 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 18:34:10.300467 21180 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:34:10.300515 21182 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 18:34:10.300587 21172 server_base.cc:1034] running on GCE node
W20260806 18:34:10.300478 21179 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 18:34:10.301151 21172 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 18:34:10.301729 21172 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 18:34:10.302908 21172 hybrid_clock.cc:648] HybridClock initialized: now 1786041250302894 us; error 25 us; skew 500 ppm
Aug 06 18:34:10 dist-test-slave-1xrh krb5kdc[20781](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041250, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.194.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 18:34:10.305523 21172 init.cc:377] Logged in from keytab as kudu/127.19.194.3@KRBTEST.COM (short username kudu)
I20260806 18:34:10.306353 21172 webserver.cc:460] Webserver started at http://127.19.194.3:34139/ using document root <none> and password file <none>
I20260806 18:34:10.306867 21172 fs_manager.cc:359] Metadata directory not provided
I20260806 18:34:10.306972 21172 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 18:34:10.307232 21172 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 18:34:10.309227 21172 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786041246094603-20232-0/minicluster-data/ts-2/data/instance:
uuid: "2fd1acdd13c3452fb8ca20f518395152"
format_stamp: "Formatted at 2026-08-06 18:34:10 on dist-test-slave-1xrh"
server_key: "5763922727641ca13a1d5dd77298bd79"
server_key_iv: "074b0939f53de20427f54803ea70d146"
server_key_version: "encryptionkey@0"
I20260806 18:34:10.309914 21172 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786041246094603-20232-0/minicluster-data/ts-2/wal/instance:
uuid: "2fd1acdd13c3452fb8ca20f518395152"
format_stamp: "Formatted at 2026-08-06 18:34:10 on dist-test-slave-1xrh"
server_key: "5763922727641ca13a1d5dd77298bd79"
server_key_iv: "074b0939f53de20427f54803ea70d146"
server_key_version: "encryptionkey@0"
I20260806 18:34:10.315375 21172 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.000s
I20260806 18:34:10.318593 21190 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 18:34:10.320386 21172 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260806 18:34:10.320506 21172 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786041246094603-20232-0/minicluster-data/ts-2/data,/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786041246094603-20232-0/minicluster-data/ts-2/wal
uuid: "2fd1acdd13c3452fb8ca20f518395152"
format_stamp: "Formatted at 2026-08-06 18:34:10 on dist-test-slave-1xrh"
server_key: "5763922727641ca13a1d5dd77298bd79"
server_key_iv: "074b0939f53de20427f54803ea70d146"
server_key_version: "encryptionkey@0"
I20260806 18:34:10.320633 21172 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786041246094603-20232-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786041246094603-20232-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786041246094603-20232-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 18:34:10.335909 21172 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 18:34:10.338881 21172 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 18:34:10.339102 21172 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 18:34:10.339736 21172 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 18:34:10.340559 21172 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 18:34:10.340606 21172 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 18:34:10.340683 21172 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 18:34:10.340714 21172 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 18:34:10.349399 21172 rpc_server.cc:307] RPC server started. Bound to: 127.19.194.3:45845
I20260806 18:34:10.349505 21311 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.194.3:45845 every 8 connection(s)
I20260806 18:34:10.350278 21172 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786041246094603-20232-0/minicluster-data/ts-2/data/info.pb
I20260806 18:34:10.357915 20232 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJW69tn/build/debug/bin/kudu as pid 21172
I20260806 18:34:10.358048 20232 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786041246094603-20232-0/minicluster-data/ts-2/wal/instance
I20260806 18:34:10.358346 20232 external_mini_cluster.cc:1428] Setting key 7d49b80d0d4e368b103777fd58b29753
Aug 06 18:34:10 dist-test-slave-1xrh krb5kdc[20781](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041250, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.194.3@KRBTEST.COM for kudu/127.19.194.62@KRBTEST.COM
I20260806 18:34:10.363425 20869 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:34:10.352119 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.194.3:56701 (local address 127.19.194.62:37651)
0806 18:34:10.352255 (+ 136us) server_negotiation.cc:206] Beginning negotiation
0806 18:34:10.352259 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 18:34:10.353110 (+ 851us) server_negotiation.cc:407] Connection header received
0806 18:34:10.354047 (+ 937us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:34:10.354050 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:34:10.354119 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:34:10.354209 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:34:10.355458 (+ 1249us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:34:10.355994 (+ 536us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:34:10.357215 (+ 1221us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:34:10.357439 (+ 224us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:34:10.357542 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:34:10.360177 (+ 2635us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:34:10.360182 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:34:10.360185 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 18:34:10.360211 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:34:10.361638 (+ 1427us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:34:10.362099 (+ 461us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:34:10.362102 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:34:10.362103 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:34:10.362176 (+ 73us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:34:10.362486 (+ 310us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:34:10.362488 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:34:10.362489 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:34:10.362661 (+ 172us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:34:10.362734 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0806 18:34:10.363209 (+ 475us) server_negotiation.cc:299] Negotiation successful
0806 18:34:10.363307 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260806 18:34:10.364269 21314 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:34:10.352456 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.194.62:37651 (local address 127.19.194.3:56701)
0806 18:34:10.352960 (+ 504us) negotiation.cc:107] Waiting for socket to connect
0806 18:34:10.353004 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0806 18:34:10.353835 (+ 831us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:34:10.354378 (+ 543us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:34:10.354384 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:34:10.354753 (+ 369us) client_negotiation.cc:189] Negotiated authn=SASL
0806 18:34:10.355302 (+ 549us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:34:10.355317 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:34:10.356136 (+ 819us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:34:10.356140 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:34:10.357079 (+ 939us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:34:10.357090 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:34:10.357607 (+ 517us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:34:10.357612 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:34:10.357773 (+ 161us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:34:10.358383 (+ 610us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 18:34:10.358400 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0806 18:34:10.360024 (+ 1624us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 18:34:10.361744 (+ 1720us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:34:10.361749 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:34:10.361760 (+ 11us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:34:10.362013 (+ 253us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:34:10.362267 (+ 254us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:34:10.362269 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:34:10.362271 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:34:10.362410 (+ 139us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:34:10.362746 (+ 336us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 18:34:10.362751 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 18:34:10.362969 (+ 218us) client_negotiation.cc:769] Sending connection context
0806 18:34:10.363167 (+ 198us) client_negotiation.cc:240] Negotiation successful
0806 18:34:10.363449 (+ 282us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":283,"thread_start_us":125,"threads_started":1}
I20260806 18:34:10.365381 21312 heartbeater.cc:347] Connected to a master server at 127.19.194.62:37651
I20260806 18:34:10.365625 21312 heartbeater.cc:464] Registering TS with master...
I20260806 18:34:10.366111 21312 heartbeater.cc:511] Master 127.19.194.62:37651 requested a full tablet report, sending...
I20260806 18:34:10.367092 20829 ts_manager.cc:194] Registered new tserver with Master: 2fd1acdd13c3452fb8ca20f518395152 (127.19.194.3:45845)
I20260806 18:34:10.367720 20829 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.19.194.3@KRBTEST.COM'} at 127.19.194.3:56701
I20260806 18:34:10.372030 20232 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260806 18:34:10.381987 20869 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:34:10.375406 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40336 (local address 127.19.194.62:37651)
0806 18:34:10.375529 (+ 123us) server_negotiation.cc:206] Beginning negotiation
0806 18:34:10.375533 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 18:34:10.375699 (+ 166us) server_negotiation.cc:407] Connection header received
0806 18:34:10.375788 (+ 89us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:34:10.375790 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:34:10.375825 (+ 35us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:34:10.375889 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:34:10.376504 (+ 615us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:34:10.376985 (+ 481us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:34:10.377735 (+ 750us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:34:10.377935 (+ 200us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:34:10.378006 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:34:10.379258 (+ 1252us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:34:10.379261 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:34:10.379263 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 18:34:10.379284 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:34:10.380653 (+ 1369us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:34:10.380989 (+ 336us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:34:10.380992 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:34:10.380993 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:34:10.381083 (+ 90us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:34:10.381302 (+ 219us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:34:10.381305 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:34:10.381306 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:34:10.381497 (+ 191us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:34:10.381610 (+ 113us) server_negotiation.cc:1035] Waiting for connection context
0806 18:34:10.381792 (+ 182us) server_negotiation.cc:299] Negotiation successful
0806 18:34:10.381878 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260806 18:34:10.386231 20829 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:40336:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260806 18:34:10.389227 20829 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 18:34:10.402949 21323 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:34:10.397126 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.194.3:45845 (local address 127.0.0.1:55376)
0806 18:34:10.397645 (+ 519us) negotiation.cc:107] Waiting for socket to connect
0806 18:34:10.397690 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0806 18:34:10.397870 (+ 180us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:34:10.399049 (+ 1179us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:34:10.399055 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:34:10.399078 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 18:34:10.399281 (+ 203us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:34:10.399289 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:34:10.400385 (+ 1096us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:34:10.400390 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:34:10.401563 (+ 1173us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:34:10.401572 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:34:10.402361 (+ 789us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:34:10.402365 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:34:10.402473 (+ 108us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:34:10.402510 (+ 37us) client_negotiation.cc:769] Sending connection context
0806 18:34:10.402639 (+ 129us) client_negotiation.cc:240] Negotiation successful
0806 18:34:10.402771 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":369,"thread_start_us":78,"threads_started":1}
I20260806 18:34:10.402956 21325 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:34:10.397295 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55376 (local address 127.19.194.3:45845)
0806 18:34:10.398490 (+ 1195us) server_negotiation.cc:206] Beginning negotiation
0806 18:34:10.398502 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0806 18:34:10.398526 (+ 24us) server_negotiation.cc:407] Connection header received
0806 18:34:10.398603 (+ 77us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:34:10.398609 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:34:10.398880 (+ 271us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:34:10.399060 (+ 180us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 18:34:10.399418 (+ 358us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:34:10.400257 (+ 839us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:34:10.401677 (+ 1420us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:34:10.402236 (+ 559us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:34:10.402359 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:34:10.402514 (+ 155us) server_negotiation.cc:1035] Waiting for connection context
0806 18:34:10.402669 (+ 155us) server_negotiation.cc:299] Negotiation successful
0806 18:34:10.402788 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1087,"thread_start_us":149,"threads_started":1}
I20260806 18:34:10.403908 21324 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:34:10.397324 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.194.1:39137 (local address 127.0.0.1:58412)
0806 18:34:10.398142 (+ 818us) negotiation.cc:107] Waiting for socket to connect
0806 18:34:10.398156 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0806 18:34:10.398242 (+ 86us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:34:10.399174 (+ 932us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:34:10.399181 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:34:10.399206 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 18:34:10.399342 (+ 136us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:34:10.399350 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:34:10.400899 (+ 1549us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:34:10.400903 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:34:10.402277 (+ 1374us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:34:10.402289 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:34:10.403500 (+ 1211us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:34:10.403504 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:34:10.403609 (+ 105us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:34:10.403633 (+ 24us) client_negotiation.cc:769] Sending connection context
0806 18:34:10.403688 (+ 55us) client_negotiation.cc:240] Negotiation successful
0806 18:34:10.403768 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":752,"mutex_wait_us":28,"thread_start_us":119,"threads_started":1}
I20260806 18:34:10.404233 21326 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:34:10.397842 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.194.2:39009 (local address 127.0.0.1:37706)
0806 18:34:10.398299 (+ 457us) negotiation.cc:107] Waiting for socket to connect
0806 18:34:10.398314 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0806 18:34:10.398405 (+ 91us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:34:10.399278 (+ 873us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:34:10.399281 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:34:10.399298 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 18:34:10.399412 (+ 114us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:34:10.399417 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:34:10.400899 (+ 1482us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:34:10.400903 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:34:10.402277 (+ 1374us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:34:10.402289 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:34:10.403833 (+ 1544us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:34:10.403836 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:34:10.403955 (+ 119us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:34:10.403980 (+ 25us) client_negotiation.cc:769] Sending connection context
0806 18:34:10.404049 (+ 69us) client_negotiation.cc:240] Negotiation successful
0806 18:34:10.404114 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":395,"mutex_wait_us":142,"thread_start_us":81,"threads_started":1}
I20260806 18:34:10.404585 21328 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:34:10.398295 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37706 (local address 127.19.194.2:39009)
0806 18:34:10.398817 (+ 522us) server_negotiation.cc:206] Beginning negotiation
0806 18:34:10.398829 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0806 18:34:10.398848 (+ 19us) server_negotiation.cc:407] Connection header received
0806 18:34:10.398911 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:34:10.398915 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:34:10.399085 (+ 170us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:34:10.399283 (+ 198us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 18:34:10.399542 (+ 259us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:34:10.400718 (+ 1176us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:34:10.402426 (+ 1708us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:34:10.403705 (+ 1279us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:34:10.403815 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:34:10.403950 (+ 135us) server_negotiation.cc:1035] Waiting for connection context
0806 18:34:10.404195 (+ 245us) server_negotiation.cc:299] Negotiation successful
0806 18:34:10.404431 (+ 236us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":425,"thread_start_us":112,"threads_started":1}
I20260806 18:34:10.404619 21327 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:34:10.398091 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58412 (local address 127.19.194.1:39137)
0806 18:34:10.398677 (+ 586us) server_negotiation.cc:206] Beginning negotiation
0806 18:34:10.398693 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0806 18:34:10.398717 (+ 24us) server_negotiation.cc:407] Connection header received
0806 18:34:10.398793 (+ 76us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:34:10.398798 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:34:10.398984 (+ 186us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:34:10.399489 (+ 505us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 18:34:10.399665 (+ 176us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:34:10.400759 (+ 1094us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:34:10.402426 (+ 1667us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:34:10.403350 (+ 924us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:34:10.404196 (+ 846us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:34:10.404340 (+ 144us) server_negotiation.cc:1035] Waiting for connection context
0806 18:34:10.404388 (+ 48us) server_negotiation.cc:299] Negotiation successful
0806 18:34:10.404464 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":469,"thread_start_us":105,"threads_started":1}
I20260806 18:34:10.406272 21241 tablet_service.cc:1467] Processing CreateTablet for tablet b93d1de2afcd48e6ada8d8a567a74610 (DEFAULT_TABLE table=test-table [id=8d19a94a148a4f8986a20ea620e2319b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 18:34:10.407524 21241 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b93d1de2afcd48e6ada8d8a567a74610. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 18:34:10.407624 20960 tablet_service.cc:1467] Processing CreateTablet for tablet b93d1de2afcd48e6ada8d8a567a74610 (DEFAULT_TABLE table=test-table [id=8d19a94a148a4f8986a20ea620e2319b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 18:34:10.408789 20960 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b93d1de2afcd48e6ada8d8a567a74610. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 18:34:10.409849 21098 tablet_service.cc:1467] Processing CreateTablet for tablet b93d1de2afcd48e6ada8d8a567a74610 (DEFAULT_TABLE table=test-table [id=8d19a94a148a4f8986a20ea620e2319b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 18:34:10.410846 21098 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b93d1de2afcd48e6ada8d8a567a74610. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 18:34:10.417667 21329 tablet_bootstrap.cc:492] T b93d1de2afcd48e6ada8d8a567a74610 P ab02a1498b104224b37835f5e62baeb3: Bootstrap starting.
I20260806 18:34:10.418033 21330 tablet_bootstrap.cc:492] T b93d1de2afcd48e6ada8d8a567a74610 P 2fd1acdd13c3452fb8ca20f518395152: Bootstrap starting.
I20260806 18:34:10.418290 21331 tablet_bootstrap.cc:492] T b93d1de2afcd48e6ada8d8a567a74610 P d5642a5cae6f42b3b2aa3a8159c7820f: Bootstrap starting.
I20260806 18:34:10.420244 21329 tablet_bootstrap.cc:654] T b93d1de2afcd48e6ada8d8a567a74610 P ab02a1498b104224b37835f5e62baeb3: Neither blocks nor log segments found. Creating new log.
I20260806 18:34:10.420272 21330 tablet_bootstrap.cc:654] T b93d1de2afcd48e6ada8d8a567a74610 P 2fd1acdd13c3452fb8ca20f518395152: Neither blocks nor log segments found. Creating new log.
I20260806 18:34:10.420715 21331 tablet_bootstrap.cc:654] T b93d1de2afcd48e6ada8d8a567a74610 P d5642a5cae6f42b3b2aa3a8159c7820f: Neither blocks nor log segments found. Creating new log.
I20260806 18:34:10.421428 21329 log.cc:824] T b93d1de2afcd48e6ada8d8a567a74610 P ab02a1498b104224b37835f5e62baeb3: Log is configured to *not* fsync() on all Append() calls
I20260806 18:34:10.421550 21331 log.cc:824] T b93d1de2afcd48e6ada8d8a567a74610 P d5642a5cae6f42b3b2aa3a8159c7820f: Log is configured to *not* fsync() on all Append() calls
I20260806 18:34:10.421550 21330 log.cc:824] T b93d1de2afcd48e6ada8d8a567a74610 P 2fd1acdd13c3452fb8ca20f518395152: Log is configured to *not* fsync() on all Append() calls
I20260806 18:34:10.423481 21331 tablet_bootstrap.cc:492] T b93d1de2afcd48e6ada8d8a567a74610 P d5642a5cae6f42b3b2aa3a8159c7820f: No bootstrap required, opened a new log
I20260806 18:34:10.423534 21329 tablet_bootstrap.cc:492] T b93d1de2afcd48e6ada8d8a567a74610 P ab02a1498b104224b37835f5e62baeb3: No bootstrap required, opened a new log
I20260806 18:34:10.423635 21331 ts_tablet_manager.cc:1397] T b93d1de2afcd48e6ada8d8a567a74610 P d5642a5cae6f42b3b2aa3a8159c7820f: Time spent bootstrapping tablet: real 0.006s user 0.005s sys 0.000s
I20260806 18:34:10.423723 21329 ts_tablet_manager.cc:1397] T b93d1de2afcd48e6ada8d8a567a74610 P ab02a1498b104224b37835f5e62baeb3: Time spent bootstrapping tablet: real 0.006s user 0.005s sys 0.000s
I20260806 18:34:10.424603 21330 tablet_bootstrap.cc:492] T b93d1de2afcd48e6ada8d8a567a74610 P 2fd1acdd13c3452fb8ca20f518395152: No bootstrap required, opened a new log
I20260806 18:34:10.424832 21330 ts_tablet_manager.cc:1397] T b93d1de2afcd48e6ada8d8a567a74610 P 2fd1acdd13c3452fb8ca20f518395152: Time spent bootstrapping tablet: real 0.007s user 0.005s sys 0.000s
I20260806 18:34:10.426584 21331 raft_consensus.cc:348] T b93d1de2afcd48e6ada8d8a567a74610 P d5642a5cae6f42b3b2aa3a8159c7820f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2fd1acdd13c3452fb8ca20f518395152" member_type: VOTER last_known_addr { host: "127.19.194.3" port: 45845 } } peers { permanent_uuid: "ab02a1498b104224b37835f5e62baeb3" member_type: VOTER last_known_addr { host: "127.19.194.1" port: 39137 } } peers { permanent_uuid: "d5642a5cae6f42b3b2aa3a8159c7820f" member_type: VOTER last_known_addr { host: "127.19.194.2" port: 39009 } }
I20260806 18:34:10.426813 21331 raft_consensus.cc:374] T b93d1de2afcd48e6ada8d8a567a74610 P d5642a5cae6f42b3b2aa3a8159c7820f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 18:34:10.426862 21331 raft_consensus.cc:729] T b93d1de2afcd48e6ada8d8a567a74610 P d5642a5cae6f42b3b2aa3a8159c7820f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d5642a5cae6f42b3b2aa3a8159c7820f, State: Initialized, Role: FOLLOWER
I20260806 18:34:10.427295 21331 consensus_queue.cc:260] T b93d1de2afcd48e6ada8d8a567a74610 P d5642a5cae6f42b3b2aa3a8159c7820f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2fd1acdd13c3452fb8ca20f518395152" member_type: VOTER last_known_addr { host: "127.19.194.3" port: 45845 } } peers { permanent_uuid: "ab02a1498b104224b37835f5e62baeb3" member_type: VOTER last_known_addr { host: "127.19.194.1" port: 39137 } } peers { permanent_uuid: "d5642a5cae6f42b3b2aa3a8159c7820f" member_type: VOTER last_known_addr { host: "127.19.194.2" port: 39009 } }
I20260806 18:34:10.427896 21167 heartbeater.cc:503] Master 127.19.194.62:37651 was elected leader, sending a full tablet report...
I20260806 18:34:10.427785 21329 raft_consensus.cc:348] T b93d1de2afcd48e6ada8d8a567a74610 P ab02a1498b104224b37835f5e62baeb3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2fd1acdd13c3452fb8ca20f518395152" member_type: VOTER last_known_addr { host: "127.19.194.3" port: 45845 } } peers { permanent_uuid: "ab02a1498b104224b37835f5e62baeb3" member_type: VOTER last_known_addr { host: "127.19.194.1" port: 39137 } } peers { permanent_uuid: "d5642a5cae6f42b3b2aa3a8159c7820f" member_type: VOTER last_known_addr { host: "127.19.194.2" port: 39009 } }
I20260806 18:34:10.428037 21329 raft_consensus.cc:374] T b93d1de2afcd48e6ada8d8a567a74610 P ab02a1498b104224b37835f5e62baeb3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 18:34:10.428144 21329 raft_consensus.cc:729] T b93d1de2afcd48e6ada8d8a567a74610 P ab02a1498b104224b37835f5e62baeb3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ab02a1498b104224b37835f5e62baeb3, State: Initialized, Role: FOLLOWER
I20260806 18:34:10.428272 21331 ts_tablet_manager.cc:1428] T b93d1de2afcd48e6ada8d8a567a74610 P d5642a5cae6f42b3b2aa3a8159c7820f: Time spent starting tablet: real 0.005s user 0.004s sys 0.000s
I20260806 18:34:10.428619 21329 consensus_queue.cc:260] T b93d1de2afcd48e6ada8d8a567a74610 P ab02a1498b104224b37835f5e62baeb3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2fd1acdd13c3452fb8ca20f518395152" member_type: VOTER last_known_addr { host: "127.19.194.3" port: 45845 } } peers { permanent_uuid: "ab02a1498b104224b37835f5e62baeb3" member_type: VOTER last_known_addr { host: "127.19.194.1" port: 39137 } } peers { permanent_uuid: "d5642a5cae6f42b3b2aa3a8159c7820f" member_type: VOTER last_known_addr { host: "127.19.194.2" port: 39009 } }
I20260806 18:34:10.429630 21329 ts_tablet_manager.cc:1428] T b93d1de2afcd48e6ada8d8a567a74610 P ab02a1498b104224b37835f5e62baeb3: Time spent starting tablet: real 0.006s user 0.006s sys 0.000s
I20260806 18:34:10.429497 21330 raft_consensus.cc:348] T b93d1de2afcd48e6ada8d8a567a74610 P 2fd1acdd13c3452fb8ca20f518395152 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2fd1acdd13c3452fb8ca20f518395152" member_type: VOTER last_known_addr { host: "127.19.194.3" port: 45845 } } peers { permanent_uuid: "ab02a1498b104224b37835f5e62baeb3" member_type: VOTER last_known_addr { host: "127.19.194.1" port: 39137 } } peers { permanent_uuid: "d5642a5cae6f42b3b2aa3a8159c7820f" member_type: VOTER last_known_addr { host: "127.19.194.2" port: 39009 } }
I20260806 18:34:10.429745 21330 raft_consensus.cc:374] T b93d1de2afcd48e6ada8d8a567a74610 P 2fd1acdd13c3452fb8ca20f518395152 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 18:34:10.429785 21330 raft_consensus.cc:729] T b93d1de2afcd48e6ada8d8a567a74610 P 2fd1acdd13c3452fb8ca20f518395152 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2fd1acdd13c3452fb8ca20f518395152, State: Initialized, Role: FOLLOWER
I20260806 18:34:10.430121 21026 heartbeater.cc:503] Master 127.19.194.62:37651 was elected leader, sending a full tablet report...
I20260806 18:34:10.430199 21330 consensus_queue.cc:260] T b93d1de2afcd48e6ada8d8a567a74610 P 2fd1acdd13c3452fb8ca20f518395152 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2fd1acdd13c3452fb8ca20f518395152" member_type: VOTER last_known_addr { host: "127.19.194.3" port: 45845 } } peers { permanent_uuid: "ab02a1498b104224b37835f5e62baeb3" member_type: VOTER last_known_addr { host: "127.19.194.1" port: 39137 } } peers { permanent_uuid: "d5642a5cae6f42b3b2aa3a8159c7820f" member_type: VOTER last_known_addr { host: "127.19.194.2" port: 39009 } }
I20260806 18:34:10.430888 21312 heartbeater.cc:503] Master 127.19.194.62:37651 was elected leader, sending a full tablet report...
I20260806 18:34:10.431151 21330 ts_tablet_manager.cc:1428] T b93d1de2afcd48e6ada8d8a567a74610 P 2fd1acdd13c3452fb8ca20f518395152: Time spent starting tablet: real 0.006s user 0.005s sys 0.000s
I20260806 18:34:10.592288 21336 raft_consensus.cc:482] T b93d1de2afcd48e6ada8d8a567a74610 P ab02a1498b104224b37835f5e62baeb3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 18:34:10.592499 21336 raft_consensus.cc:504] T b93d1de2afcd48e6ada8d8a567a74610 P ab02a1498b104224b37835f5e62baeb3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2fd1acdd13c3452fb8ca20f518395152" member_type: VOTER last_known_addr { host: "127.19.194.3" port: 45845 } } peers { permanent_uuid: "ab02a1498b104224b37835f5e62baeb3" member_type: VOTER last_known_addr { host: "127.19.194.1" port: 39137 } } peers { permanent_uuid: "d5642a5cae6f42b3b2aa3a8159c7820f" member_type: VOTER last_known_addr { host: "127.19.194.2" port: 39009 } }
I20260806 18:34:10.593608 21336 leader_election.cc:288] T b93d1de2afcd48e6ada8d8a567a74610 P ab02a1498b104224b37835f5e62baeb3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 2fd1acdd13c3452fb8ca20f518395152 (127.19.194.3:45845), d5642a5cae6f42b3b2aa3a8159c7820f (127.19.194.2:39009)
I20260806 18:34:10.597664 21328 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:34:10.594042 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.194.1:44643 (local address 127.19.194.2:39009)
0806 18:34:10.594173 (+ 131us) server_negotiation.cc:206] Beginning negotiation
0806 18:34:10.594177 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 18:34:10.594407 (+ 230us) server_negotiation.cc:407] Connection header received
0806 18:34:10.594526 (+ 119us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:34:10.594531 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:34:10.594586 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:34:10.594664 (+ 78us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 18:34:10.594969 (+ 305us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:34:10.595575 (+ 606us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:34:10.596671 (+ 1096us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:34:10.597067 (+ 396us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:34:10.597158 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:34:10.597230 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0806 18:34:10.597455 (+ 225us) server_negotiation.cc:299] Negotiation successful
0806 18:34:10.597535 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260806 18:34:10.597680 21325 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:34:10.593974 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.194.1:44451 (local address 127.19.194.3:45845)
0806 18:34:10.594119 (+ 145us) server_negotiation.cc:206] Beginning negotiation
0806 18:34:10.594126 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 18:34:10.594284 (+ 158us) server_negotiation.cc:407] Connection header received
0806 18:34:10.594391 (+ 107us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:34:10.594394 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:34:10.594450 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:34:10.594537 (+ 87us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 18:34:10.594845 (+ 308us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:34:10.595590 (+ 745us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:34:10.596671 (+ 1081us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:34:10.597079 (+ 408us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:34:10.597254 (+ 175us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:34:10.597314 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0806 18:34:10.597499 (+ 185us) server_negotiation.cc:299] Negotiation successful
0806 18:34:10.597579 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260806 18:34:10.597671 21339 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:34:10.593857 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.194.2:39009 (local address 127.19.194.1:44643)
0806 18:34:10.594293 (+ 436us) negotiation.cc:107] Waiting for socket to connect
0806 18:34:10.594309 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0806 18:34:10.594417 (+ 108us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:34:10.594697 (+ 280us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:34:10.594700 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:34:10.594714 (+ 14us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 18:34:10.594845 (+ 131us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:34:10.594852 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:34:10.595679 (+ 827us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:34:10.595683 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:34:10.596536 (+ 853us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:34:10.596546 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:34:10.597209 (+ 663us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:34:10.597215 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:34:10.597314 (+ 99us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:34:10.597344 (+ 30us) client_negotiation.cc:769] Sending connection context
0806 18:34:10.597415 (+ 71us) client_negotiation.cc:240] Negotiation successful
0806 18:34:10.597503 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":356,"mutex_wait_us":1,"thread_start_us":115,"threads_started":1}
I20260806 18:34:10.597683 21338 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:34:10.593804 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.194.3:45845 (local address 127.19.194.1:44451)
0806 18:34:10.594174 (+ 370us) negotiation.cc:107] Waiting for socket to connect
0806 18:34:10.594191 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0806 18:34:10.594295 (+ 104us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:34:10.594552 (+ 257us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:34:10.594555 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:34:10.594572 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 18:34:10.594718 (+ 146us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:34:10.594725 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:34:10.595717 (+ 992us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:34:10.595721 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:34:10.596541 (+ 820us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:34:10.596550 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:34:10.597261 (+ 711us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:34:10.597265 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:34:10.597359 (+ 94us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:34:10.597376 (+ 17us) client_negotiation.cc:769] Sending connection context
0806 18:34:10.597479 (+ 103us) client_negotiation.cc:240] Negotiation successful
0806 18:34:10.597542 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":303,"mutex_wait_us":32,"thread_start_us":114,"threads_started":1}
I20260806 18:34:10.598342 21119 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b93d1de2afcd48e6ada8d8a567a74610" candidate_uuid: "ab02a1498b104224b37835f5e62baeb3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d5642a5cae6f42b3b2aa3a8159c7820f" is_pre_election: true
I20260806 18:34:10.598682 21119 raft_consensus.cc:2454] T b93d1de2afcd48e6ada8d8a567a74610 P d5642a5cae6f42b3b2aa3a8159c7820f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ab02a1498b104224b37835f5e62baeb3 in term 0.
I20260806 18:34:10.598683 21262 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b93d1de2afcd48e6ada8d8a567a74610" candidate_uuid: "ab02a1498b104224b37835f5e62baeb3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2fd1acdd13c3452fb8ca20f518395152" is_pre_election: true
I20260806 18:34:10.599004 21262 raft_consensus.cc:2454] T b93d1de2afcd48e6ada8d8a567a74610 P 2fd1acdd13c3452fb8ca20f518395152 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ab02a1498b104224b37835f5e62baeb3 in term 0.
I20260806 18:34:10.599197 20910 leader_election.cc:302] T b93d1de2afcd48e6ada8d8a567a74610 P ab02a1498b104224b37835f5e62baeb3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ab02a1498b104224b37835f5e62baeb3, d5642a5cae6f42b3b2aa3a8159c7820f; no voters:
I20260806 18:34:10.599479 21336 raft_consensus.cc:2781] T b93d1de2afcd48e6ada8d8a567a74610 P ab02a1498b104224b37835f5e62baeb3 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 18:34:10.599566 21336 raft_consensus.cc:482] T b93d1de2afcd48e6ada8d8a567a74610 P ab02a1498b104224b37835f5e62baeb3 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 18:34:10.599598 21336 raft_consensus.cc:3037] T b93d1de2afcd48e6ada8d8a567a74610 P ab02a1498b104224b37835f5e62baeb3 [term 0 FOLLOWER]: Advancing to term 1
I20260806 18:34:10.600665 21336 raft_consensus.cc:504] T b93d1de2afcd48e6ada8d8a567a74610 P ab02a1498b104224b37835f5e62baeb3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2fd1acdd13c3452fb8ca20f518395152" member_type: VOTER last_known_addr { host: "127.19.194.3" port: 45845 } } peers { permanent_uuid: "ab02a1498b104224b37835f5e62baeb3" member_type: VOTER last_known_addr { host: "127.19.194.1" port: 39137 } } peers { permanent_uuid: "d5642a5cae6f42b3b2aa3a8159c7820f" member_type: VOTER last_known_addr { host: "127.19.194.2" port: 39009 } }
I20260806 18:34:10.601053 21336 leader_election.cc:288] T b93d1de2afcd48e6ada8d8a567a74610 P ab02a1498b104224b37835f5e62baeb3 [CANDIDATE]: Term 1 election: Requested vote from peers 2fd1acdd13c3452fb8ca20f518395152 (127.19.194.3:45845), d5642a5cae6f42b3b2aa3a8159c7820f (127.19.194.2:39009)
W20260806 18:34:10.601263 21313 tablet.cc:2369] T b93d1de2afcd48e6ada8d8a567a74610 P 2fd1acdd13c3452fb8ca20f518395152: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 18:34:10.601373 21262 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b93d1de2afcd48e6ada8d8a567a74610" candidate_uuid: "ab02a1498b104224b37835f5e62baeb3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2fd1acdd13c3452fb8ca20f518395152"
I20260806 18:34:10.601380 21119 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b93d1de2afcd48e6ada8d8a567a74610" candidate_uuid: "ab02a1498b104224b37835f5e62baeb3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d5642a5cae6f42b3b2aa3a8159c7820f"
I20260806 18:34:10.601465 21262 raft_consensus.cc:3037] T b93d1de2afcd48e6ada8d8a567a74610 P 2fd1acdd13c3452fb8ca20f518395152 [term 0 FOLLOWER]: Advancing to term 1
I20260806 18:34:10.601495 21119 raft_consensus.cc:3037] T b93d1de2afcd48e6ada8d8a567a74610 P d5642a5cae6f42b3b2aa3a8159c7820f [term 0 FOLLOWER]: Advancing to term 1
I20260806 18:34:10.602662 21119 raft_consensus.cc:2454] T b93d1de2afcd48e6ada8d8a567a74610 P d5642a5cae6f42b3b2aa3a8159c7820f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ab02a1498b104224b37835f5e62baeb3 in term 1.
I20260806 18:34:10.602802 21262 raft_consensus.cc:2454] T b93d1de2afcd48e6ada8d8a567a74610 P 2fd1acdd13c3452fb8ca20f518395152 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ab02a1498b104224b37835f5e62baeb3 in term 1.
I20260806 18:34:10.603073 20910 leader_election.cc:302] T b93d1de2afcd48e6ada8d8a567a74610 P ab02a1498b104224b37835f5e62baeb3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ab02a1498b104224b37835f5e62baeb3, d5642a5cae6f42b3b2aa3a8159c7820f; no voters:
I20260806 18:34:10.603263 21336 raft_consensus.cc:2781] T b93d1de2afcd48e6ada8d8a567a74610 P ab02a1498b104224b37835f5e62baeb3 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 18:34:10.603500 21336 raft_consensus.cc:686] T b93d1de2afcd48e6ada8d8a567a74610 P ab02a1498b104224b37835f5e62baeb3 [term 1 LEADER]: Becoming Leader. State: Replica: ab02a1498b104224b37835f5e62baeb3, State: Running, Role: LEADER
I20260806 18:34:10.603886 21336 consensus_queue.cc:237] T b93d1de2afcd48e6ada8d8a567a74610 P ab02a1498b104224b37835f5e62baeb3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2fd1acdd13c3452fb8ca20f518395152" member_type: VOTER last_known_addr { host: "127.19.194.3" port: 45845 } } peers { permanent_uuid: "ab02a1498b104224b37835f5e62baeb3" member_type: VOTER last_known_addr { host: "127.19.194.1" port: 39137 } } peers { permanent_uuid: "d5642a5cae6f42b3b2aa3a8159c7820f" member_type: VOTER last_known_addr { host: "127.19.194.2" port: 39009 } }
I20260806 18:34:10.607492 20828 catalog_manager.cc:5495] T b93d1de2afcd48e6ada8d8a567a74610 P ab02a1498b104224b37835f5e62baeb3 reported cstate change: term changed from 0 to 1, leader changed from <none> to ab02a1498b104224b37835f5e62baeb3 (127.19.194.1). New cstate: current_term: 1 leader_uuid: "ab02a1498b104224b37835f5e62baeb3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2fd1acdd13c3452fb8ca20f518395152" member_type: VOTER last_known_addr { host: "127.19.194.3" port: 45845 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ab02a1498b104224b37835f5e62baeb3" member_type: VOTER last_known_addr { host: "127.19.194.1" port: 39137 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d5642a5cae6f42b3b2aa3a8159c7820f" member_type: VOTER last_known_addr { host: "127.19.194.2" port: 39009 } health_report { overall_health: UNKNOWN } } }
Aug 06 18:34:10 dist-test-slave-1xrh krb5kdc[20781](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041249, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.19.194.1@KRBTEST.COM
W20260806 18:34:10.638916 21168 tablet.cc:2369] T b93d1de2afcd48e6ada8d8a567a74610 P d5642a5cae6f42b3b2aa3a8159c7820f: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 18:34:10.642130 21327 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:34:10.633191 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58426 (local address 127.19.194.1:39137)
0806 18:34:10.633371 (+ 180us) server_negotiation.cc:206] Beginning negotiation
0806 18:34:10.633376 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 18:34:10.633516 (+ 140us) server_negotiation.cc:407] Connection header received
0806 18:34:10.633680 (+ 164us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:34:10.633684 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:34:10.633736 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:34:10.633823 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:34:10.634560 (+ 737us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:34:10.635058 (+ 498us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:34:10.635957 (+ 899us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:34:10.636217 (+ 260us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:34:10.636333 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:34:10.638805 (+ 2472us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:34:10.638817 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:34:10.638830 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 18:34:10.638852 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:34:10.640557 (+ 1705us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:34:10.640943 (+ 386us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:34:10.640948 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:34:10.640949 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:34:10.641037 (+ 88us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:34:10.641252 (+ 215us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:34:10.641254 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:34:10.641255 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:34:10.641483 (+ 228us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:34:10.641634 (+ 151us) server_negotiation.cc:1035] Waiting for connection context
0806 18:34:10.641820 (+ 186us) server_negotiation.cc:299] Negotiation successful
0806 18:34:10.641915 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":87}
Aug 06 18:34:10 dist-test-slave-1xrh krb5kdc[20781](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041250, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 06 18:34:10 dist-test-slave-1xrh krb5kdc[20781](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041250, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.19.194.1@KRBTEST.COM
I20260806 18:34:10.669325 21327 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:34:10.661257 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58436 (local address 127.19.194.1:39137)
0806 18:34:10.661421 (+ 164us) server_negotiation.cc:206] Beginning negotiation
0806 18:34:10.661425 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 18:34:10.661485 (+ 60us) server_negotiation.cc:407] Connection header received
0806 18:34:10.661621 (+ 136us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:34:10.661623 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:34:10.661663 (+ 40us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:34:10.661771 (+ 108us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:34:10.662434 (+ 663us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:34:10.662948 (+ 514us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:34:10.663861 (+ 913us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:34:10.664099 (+ 238us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:34:10.664207 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:34:10.666550 (+ 2343us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:34:10.666553 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:34:10.666555 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 18:34:10.666577 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:34:10.667835 (+ 1258us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:34:10.668207 (+ 372us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:34:10.668210 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:34:10.668211 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:34:10.668281 (+ 70us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:34:10.668506 (+ 225us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:34:10.668508 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:34:10.668509 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:34:10.668664 (+ 155us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:34:10.668833 (+ 169us) server_negotiation.cc:1035] Waiting for connection context
0806 18:34:10.669120 (+ 287us) server_negotiation.cc:299] Negotiation successful
0806 18:34:10.669215 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":72}
W20260806 18:34:10.670040 21027 tablet.cc:2369] T b93d1de2afcd48e6ada8d8a567a74610 P ab02a1498b104224b37835f5e62baeb3: Can't schedule compaction. Clean time has not been advanced past its initial value.
Aug 06 18:34:10 dist-test-slave-1xrh krb5kdc[20781](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041250, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 06 18:34:10 dist-test-slave-1xrh krb5kdc[20781](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041250, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.19.194.1@KRBTEST.COM
I20260806 18:34:10.693642 21327 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:34:10.685764 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58442 (local address 127.19.194.1:39137)
0806 18:34:10.685893 (+ 129us) server_negotiation.cc:206] Beginning negotiation
0806 18:34:10.685897 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 18:34:10.686055 (+ 158us) server_negotiation.cc:407] Connection header received
0806 18:34:10.686194 (+ 139us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:34:10.686218 (+ 24us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:34:10.686258 (+ 40us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:34:10.686337 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:34:10.687016 (+ 679us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:34:10.687555 (+ 539us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:34:10.688347 (+ 792us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:34:10.688581 (+ 234us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:34:10.688683 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:34:10.690819 (+ 2136us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:34:10.690823 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:34:10.690825 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 18:34:10.690846 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:34:10.692057 (+ 1211us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:34:10.692458 (+ 401us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:34:10.692461 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:34:10.692462 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:34:10.692538 (+ 76us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:34:10.692800 (+ 262us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:34:10.692804 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:34:10.692805 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:34:10.692956 (+ 151us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:34:10.693126 (+ 170us) server_negotiation.cc:1035] Waiting for connection context
0806 18:34:10.693350 (+ 224us) server_negotiation.cc:299] Negotiation successful
0806 18:34:10.693476 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
W20260806 18:34:10.694902 20939 tablet_service.cc:2908] Rejecting scan request for tablet b93d1de2afcd48e6ada8d8a567a74610: Uninitialized: clean time has not yet been initialized
I20260806 18:34:10.696903 20232 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJW69tn/build/debug/bin/kudu with pid 20892
I20260806 18:34:10.704667 20232 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJW69tn/build/debug/bin/kudu with pid 21031
I20260806 18:34:10.711262 20232 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJW69tn/build/debug/bin/kudu with pid 21172
I20260806 18:34:10.719300 20232 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJW69tn/build/debug/bin/kudu with pid 20796
2026-08-06T18:34:10Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizationOnChecksum (2417 ms)
[ RUN ] SecurityITest.SmokeTestAsAuthorizedUser
Loading random data
Initializing database '/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786041246094603-20232-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 18:34:10 dist-test-slave-1xrh krb5kdc[21356](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 18:34:10 dist-test-slave-1xrh krb5kdc[21356](info): set up 2 sockets
Aug 06 18:34:10 dist-test-slave-1xrh krb5kdc[21356](info): commencing operation
krb5kdc: starting...
W20260806 18:34:11.788563 20232 mini_kdc.cc:121] Time spent starting KDC: real 1.043s user 0.003s sys 0.003s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786041246094603-20232-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786041246094603-20232-0/krb5kdc/test-user.keytab.
Aug 06 18:34:11 dist-test-slave-1xrh krb5kdc[21356](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041251, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T18:34:11Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T18:34:11Z Disabled control of system clock
WARNING: no policy specified for kudu/127.19.194.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.19.194.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.19.194.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786041246094603-20232-0/krb5kdc/kudu_127.19.194.62.keytab.
Entry for principal kudu/127.19.194.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786041246094603-20232-0/krb5kdc/kudu_127.19.194.62.keytab.
I20260806 18:34:11.907668 20232 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJW69tn/build/debug/bin/kudu
/tmp/dist-test-taskJW69tn/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786041246094603-20232-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786041246094603-20232-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786041246094603-20232-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786041246094603-20232-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.19.194.62:34683
--webserver_interface=127.19.194.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786041246094603-20232-0/krb5kdc/kudu_127.19.194.62.keytab
--principal=kudu/127.19.194.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.19.194.20:40719
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.19.194.62:34683
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786041246094603-20232-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786041246094603-20232-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786041246094603-20232-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 18:34:12.015667 21370 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 18:34:12.015918 21370 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 18:34:12.015956 21370 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 18:34:12.019871 21370 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 18:34:12.019918 21370 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260806 18:34:12.019935 21370 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 18:34:12.019955 21370 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 18:34:12.019994 21370 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 18:34:12.020046 21370 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 18:34:12.025200 21370 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.194.20:40719
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786041246094603-20232-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786041246094603-20232-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.19.194.62:34683
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786041246094603-20232-0/krb5kdc/kudu_127.19.194.62.keytab
--principal=kudu/127.19.194.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.19.194.62:34683
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786041246094603-20232-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.19.194.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21370
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786041246094603-20232-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 17:59:13 UTC on ccycloud.quasar-wvsnzx.root.comops.site
build id 4571028
I20260806 18:34:12.026113 21370 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 18:34:12.027081 21370 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 18:34:12.035111 21380 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:34:12.035112 21377 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:34:12.035204 21376 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 18:34:12.035367 21370 server_base.cc:1034] running on GCE node
I20260806 18:34:12.036028 21370 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 18:34:12.037061 21370 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 18:34:12.038244 21370 hybrid_clock.cc:648] HybridClock initialized: now 1786041252038232 us; error 26 us; skew 500 ppm
Aug 06 18:34:12 dist-test-slave-1xrh krb5kdc[21356](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041252, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.194.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 18:34:12.047309 21370 init.cc:377] Logged in from keytab as kudu/127.19.194.62@KRBTEST.COM (short username kudu)
I20260806 18:34:12.048223 21370 webserver.cc:460] Webserver started at http://127.19.194.62:34295/ using document root <none> and password file <none>
I20260806 18:34:12.048774 21370 fs_manager.cc:359] Metadata directory not provided
I20260806 18:34:12.048863 21370 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 18:34:12.049078 21370 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 18:34:12.051003 21370 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786041246094603-20232-0/minicluster-data/master-0/data/instance:
uuid: "4bfe62b1286a4f54925c8a7b156a6adf"
format_stamp: "Formatted at 2026-08-06 18:34:12 on dist-test-slave-1xrh"
server_key: "61761ab4edb662356a191beb357cb248"
server_key_iv: "ba135dd15fa94840949a1f5857aa212b"
server_key_version: "encryptionkey@0"
I20260806 18:34:12.051621 21370 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786041246094603-20232-0/minicluster-data/master-0/wal/instance:
uuid: "4bfe62b1286a4f54925c8a7b156a6adf"
format_stamp: "Formatted at 2026-08-06 18:34:12 on dist-test-slave-1xrh"
server_key: "61761ab4edb662356a191beb357cb248"
server_key_iv: "ba135dd15fa94840949a1f5857aa212b"
server_key_version: "encryptionkey@0"
I20260806 18:34:12.055116 21370 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.000s
I20260806 18:34:12.057395 21387 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 18:34:12.058494 21370 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.000s sys 0.002s
I20260806 18:34:12.058640 21370 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786041246094603-20232-0/minicluster-data/master-0/data,/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786041246094603-20232-0/minicluster-data/master-0/wal
uuid: "4bfe62b1286a4f54925c8a7b156a6adf"
format_stamp: "Formatted at 2026-08-06 18:34:12 on dist-test-slave-1xrh"
server_key: "61761ab4edb662356a191beb357cb248"
server_key_iv: "ba135dd15fa94840949a1f5857aa212b"
server_key_version: "encryptionkey@0"
I20260806 18:34:12.058743 21370 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786041246094603-20232-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786041246094603-20232-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786041246094603-20232-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 18:34:12.087337 21370 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 18:34:12.089984 21370 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 18:34:12.090173 21370 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 18:34:12.098286 21370 rpc_server.cc:307] RPC server started. Bound to: 127.19.194.62:34683
I20260806 18:34:12.098312 21455 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.194.62:34683 every 8 connection(s)
I20260806 18:34:12.099164 21370 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786041246094603-20232-0/minicluster-data/master-0/data/info.pb
I20260806 18:34:12.101694 21456 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 18:34:12.103060 20232 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJW69tn/build/debug/bin/kudu as pid 21370
I20260806 18:34:12.103204 20232 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786041246094603-20232-0/minicluster-data/master-0/wal/instance
I20260806 18:34:12.103452 20232 external_mini_cluster.cc:1428] Setting key 4b5c309ec79c481f403331c11f569862
I20260806 18:34:12.107422 21456 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4bfe62b1286a4f54925c8a7b156a6adf: Bootstrap starting.
I20260806 18:34:12.109787 21456 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 4bfe62b1286a4f54925c8a7b156a6adf: Neither blocks nor log segments found. Creating new log.
I20260806 18:34:12.110672 21456 log.cc:824] T 00000000000000000000000000000000 P 4bfe62b1286a4f54925c8a7b156a6adf: Log is configured to *not* fsync() on all Append() calls
Aug 06 18:34:12 dist-test-slave-1xrh krb5kdc[21356](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041251, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.19.194.62@KRBTEST.COM
I20260806 18:34:12.112907 21456 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4bfe62b1286a4f54925c8a7b156a6adf: No bootstrap required, opened a new log
I20260806 18:34:12.115875 21456 raft_consensus.cc:348] T 00000000000000000000000000000000 P 4bfe62b1286a4f54925c8a7b156a6adf [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4bfe62b1286a4f54925c8a7b156a6adf" member_type: VOTER last_known_addr { host: "127.19.194.62" port: 34683 } }
I20260806 18:34:12.116130 21456 raft_consensus.cc:374] T 00000000000000000000000000000000 P 4bfe62b1286a4f54925c8a7b156a6adf [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 18:34:12.116211 21456 raft_consensus.cc:729] T 00000000000000000000000000000000 P 4bfe62b1286a4f54925c8a7b156a6adf [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4bfe62b1286a4f54925c8a7b156a6adf, State: Initialized, Role: FOLLOWER
I20260806 18:34:12.116662 21456 consensus_queue.cc:260] T 00000000000000000000000000000000 P 4bfe62b1286a4f54925c8a7b156a6adf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4bfe62b1286a4f54925c8a7b156a6adf" member_type: VOTER last_known_addr { host: "127.19.194.62" port: 34683 } }
I20260806 18:34:12.116811 21456 raft_consensus.cc:388] T 00000000000000000000000000000000 P 4bfe62b1286a4f54925c8a7b156a6adf [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 18:34:12.116876 21456 raft_consensus.cc:482] T 00000000000000000000000000000000 P 4bfe62b1286a4f54925c8a7b156a6adf [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 18:34:12.116977 21456 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 4bfe62b1286a4f54925c8a7b156a6adf [term 0 FOLLOWER]: Advancing to term 1
I20260806 18:34:12.117879 21456 raft_consensus.cc:504] T 00000000000000000000000000000000 P 4bfe62b1286a4f54925c8a7b156a6adf [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4bfe62b1286a4f54925c8a7b156a6adf" member_type: VOTER last_known_addr { host: "127.19.194.62" port: 34683 } }
I20260806 18:34:12.118214 21456 leader_election.cc:302] T 00000000000000000000000000000000 P 4bfe62b1286a4f54925c8a7b156a6adf [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 4bfe62b1286a4f54925c8a7b156a6adf; no voters:
I20260806 18:34:12.118522 21456 leader_election.cc:288] T 00000000000000000000000000000000 P 4bfe62b1286a4f54925c8a7b156a6adf [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 18:34:12.118654 21462 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 4bfe62b1286a4f54925c8a7b156a6adf [term 1 FOLLOWER]: Leader election won for term 1
I20260806 18:34:12.118856 21462 raft_consensus.cc:686] T 00000000000000000000000000000000 P 4bfe62b1286a4f54925c8a7b156a6adf [term 1 LEADER]: Becoming Leader. State: Replica: 4bfe62b1286a4f54925c8a7b156a6adf, State: Running, Role: LEADER
I20260806 18:34:12.118873 21460 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:34:12.104818 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47864 (local address 127.19.194.62:34683)
0806 18:34:12.105247 (+ 429us) server_negotiation.cc:206] Beginning negotiation
0806 18:34:12.105261 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0806 18:34:12.105295 (+ 34us) server_negotiation.cc:407] Connection header received
0806 18:34:12.105903 (+ 608us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:34:12.105918 (+ 15us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:34:12.106215 (+ 297us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:34:12.106542 (+ 327us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:34:12.107342 (+ 800us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:34:12.108293 (+ 951us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:34:12.109298 (+ 1005us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:34:12.110006 (+ 708us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:34:12.110205 (+ 199us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:34:12.113218 (+ 3013us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:34:12.113223 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:34:12.113234 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 18:34:12.113261 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:34:12.115645 (+ 2384us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:34:12.116100 (+ 455us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:34:12.116106 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:34:12.116110 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:34:12.116312 (+ 202us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:34:12.116607 (+ 295us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:34:12.116610 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:34:12.116612 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:34:12.117095 (+ 483us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:34:12.117296 (+ 201us) server_negotiation.cc:1035] Waiting for connection context
0806 18:34:12.117620 (+ 324us) server_negotiation.cc:299] Negotiation successful
0806 18:34:12.117897 (+ 277us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":300,"thread_start_us":152,"threads_started":1}
I20260806 18:34:12.119227 21462 consensus_queue.cc:237] T 00000000000000000000000000000000 P 4bfe62b1286a4f54925c8a7b156a6adf [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4bfe62b1286a4f54925c8a7b156a6adf" member_type: VOTER last_known_addr { host: "127.19.194.62" port: 34683 } }
I20260806 18:34:12.119802 21456 sys_catalog.cc:564] T 00000000000000000000000000000000 P 4bfe62b1286a4f54925c8a7b156a6adf [sys.catalog]: configured and running, proceeding with master startup.
I20260806 18:34:12.120369 21462 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4bfe62b1286a4f54925c8a7b156a6adf [sys.catalog]: SysCatalogTable state changed. Reason: New leader 4bfe62b1286a4f54925c8a7b156a6adf. Latest consensus state: current_term: 1 leader_uuid: "4bfe62b1286a4f54925c8a7b156a6adf" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4bfe62b1286a4f54925c8a7b156a6adf" member_type: VOTER last_known_addr { host: "127.19.194.62" port: 34683 } } }
I20260806 18:34:12.120486 21462 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4bfe62b1286a4f54925c8a7b156a6adf [sys.catalog]: This master's current role is: LEADER
I20260806 18:34:12.120956 21463 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4bfe62b1286a4f54925c8a7b156a6adf [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "4bfe62b1286a4f54925c8a7b156a6adf" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4bfe62b1286a4f54925c8a7b156a6adf" member_type: VOTER last_known_addr { host: "127.19.194.62" port: 34683 } } }
I20260806 18:34:12.121148 21463 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4bfe62b1286a4f54925c8a7b156a6adf [sys.catalog]: This master's current role is: LEADER
I20260806 18:34:12.121059 21469 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 18:34:12.124306 21469 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 18:34:12.129526 21469 catalog_manager.cc:1347] Generated new cluster ID: 3bf31800b5ed4773802280e8a45f540b
I20260806 18:34:12.129585 21469 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 18:34:12.138993 21469 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 18:34:12.139829 21469 catalog_manager.cc:1504] Loading token signing keys...
I20260806 18:34:12.146397 21469 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 4bfe62b1286a4f54925c8a7b156a6adf: Generated new TSK 0
I20260806 18:34:12.146996 21469 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.19.194.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.19.194.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.19.194.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786041246094603-20232-0/krb5kdc/kudu_127.19.194.1.keytab.
Entry for principal kudu/127.19.194.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786041246094603-20232-0/krb5kdc/kudu_127.19.194.1.keytab.
I20260806 18:34:12.176406 20232 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJW69tn/build/debug/bin/kudu
/tmp/dist-test-taskJW69tn/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786041246094603-20232-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786041246094603-20232-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786041246094603-20232-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786041246094603-20232-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.19.194.1:0
--local_ip_for_outbound_sockets=127.19.194.1
--webserver_interface=127.19.194.1
--webserver_port=0
--tserver_master_addrs=127.19.194.62:34683
--keytab_file=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786041246094603-20232-0/krb5kdc/kudu_127.19.194.1.keytab
--principal=kudu/127.19.194.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.19.194.20:40719
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786041246094603-20232-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786041246094603-20232-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786041246094603-20232-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 18:34:12.284051 21487 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 18:34:12.284279 21487 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 18:34:12.284320 21487 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 18:34:12.288102 21487 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 18:34:12.288148 21487 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 18:34:12.288183 21487 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260806 18:34:12.288213 21487 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.19.194.1
I20260806 18:34:12.293103 21487 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.194.20:40719
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786041246094603-20232-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786041246094603-20232-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786041246094603-20232-0/krb5kdc/kudu_127.19.194.1.keytab
--principal=kudu/127.19.194.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.19.194.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786041246094603-20232-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.19.194.1
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.19.194.62:34683
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21487
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.19.194.1
--log_dir=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786041246094603-20232-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 17:59:13 UTC on ccycloud.quasar-wvsnzx.root.comops.site
build id 4571028
I20260806 18:34:12.293993 21487 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 18:34:12.294871 21487 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260806 18:34:12.304221 21487 server_base.cc:1034] running on GCE node
W20260806 18:34:12.304140 21492 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:34:12.304140 21496 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:34:12.304144 21493 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 18:34:12.304721 21487 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 18:34:12.305320 21487 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 18:34:12.306504 21487 hybrid_clock.cc:648] HybridClock initialized: now 1786041252306491 us; error 29 us; skew 500 ppm
Aug 06 18:34:12 dist-test-slave-1xrh krb5kdc[21356](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041252, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.194.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 18:34:12.309095 21487 init.cc:377] Logged in from keytab as kudu/127.19.194.1@KRBTEST.COM (short username kudu)
I20260806 18:34:12.309953 21487 webserver.cc:460] Webserver started at http://127.19.194.1:43129/ using document root <none> and password file <none>
I20260806 18:34:12.310458 21487 fs_manager.cc:359] Metadata directory not provided
I20260806 18:34:12.310541 21487 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 18:34:12.310740 21487 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 18:34:12.312690 21487 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786041246094603-20232-0/minicluster-data/ts-0/data/instance:
uuid: "ef292515d9c349a8a04b90e9476e31cd"
format_stamp: "Formatted at 2026-08-06 18:34:12 on dist-test-slave-1xrh"
server_key: "cfff837da58cb26944554e87136fbc85"
server_key_iv: "04d5b7fa29ad922817aafa33a7d69e87"
server_key_version: "encryptionkey@0"
I20260806 18:34:12.313242 21487 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786041246094603-20232-0/minicluster-data/ts-0/wal/instance:
uuid: "ef292515d9c349a8a04b90e9476e31cd"
format_stamp: "Formatted at 2026-08-06 18:34:12 on dist-test-slave-1xrh"
server_key: "cfff837da58cb26944554e87136fbc85"
server_key_iv: "04d5b7fa29ad922817aafa33a7d69e87"
server_key_version: "encryptionkey@0"
I20260806 18:34:12.316807 21487 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.002s
I20260806 18:34:12.319015 21505 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 18:34:12.320192 21487 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260806 18:34:12.320331 21487 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786041246094603-20232-0/minicluster-data/ts-0/data,/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786041246094603-20232-0/minicluster-data/ts-0/wal
uuid: "ef292515d9c349a8a04b90e9476e31cd"
format_stamp: "Formatted at 2026-08-06 18:34:12 on dist-test-slave-1xrh"
server_key: "cfff837da58cb26944554e87136fbc85"
server_key_iv: "04d5b7fa29ad922817aafa33a7d69e87"
server_key_version: "encryptionkey@0"
I20260806 18:34:12.320493 21487 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786041246094603-20232-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786041246094603-20232-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786041246094603-20232-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 18:34:12.348598 21487 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 18:34:12.351459 21487 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 18:34:12.351641 21487 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 18:34:12.353034 21487 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 18:34:12.353098 21487 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 18:34:12.353164 21487 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 18:34:12.353207 21487 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 06 18:34:12 dist-test-slave-1xrh krb5kdc[21356](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041252, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.194.1@KRBTEST.COM for kudu/127.19.194.62@KRBTEST.COM
I20260806 18:34:12.364351 21487 rpc_server.cc:307] RPC server started. Bound to: 127.19.194.1:39693
I20260806 18:34:12.364728 21625 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.194.1:39693 every 8 connection(s)
I20260806 18:34:12.365290 21487 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786041246094603-20232-0/minicluster-data/ts-0/data/info.pb
I20260806 18:34:12.367549 21460 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:34:12.353737 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.194.1:51305 (local address 127.19.194.62:34683)
0806 18:34:12.353885 (+ 148us) server_negotiation.cc:206] Beginning negotiation
0806 18:34:12.353889 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 18:34:12.355518 (+ 1629us) server_negotiation.cc:407] Connection header received
0806 18:34:12.356752 (+ 1234us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:34:12.356756 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:34:12.356821 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:34:12.356926 (+ 105us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:34:12.358140 (+ 1214us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:34:12.359029 (+ 889us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:34:12.360239 (+ 1210us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:34:12.360677 (+ 438us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:34:12.360798 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:34:12.363654 (+ 2856us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:34:12.363660 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:34:12.363667 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 18:34:12.363702 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:34:12.365390 (+ 1688us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:34:12.366051 (+ 661us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:34:12.366055 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:34:12.366056 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:34:12.366148 (+ 92us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:34:12.366488 (+ 340us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:34:12.366491 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:34:12.366493 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:34:12.366667 (+ 174us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:34:12.366771 (+ 104us) server_negotiation.cc:1035] Waiting for connection context
0806 18:34:12.367257 (+ 486us) server_negotiation.cc:299] Negotiation successful
0806 18:34:12.367381 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260806 18:34:12.368208 21514 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:34:12.354831 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.194.62:34683 (local address 127.19.194.1:51305)
0806 18:34:12.355377 (+ 546us) negotiation.cc:107] Waiting for socket to connect
0806 18:34:12.355420 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0806 18:34:12.356547 (+ 1127us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:34:12.357124 (+ 577us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:34:12.357131 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:34:12.357451 (+ 320us) client_negotiation.cc:189] Negotiated authn=SASL
0806 18:34:12.357892 (+ 441us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:34:12.357904 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:34:12.359200 (+ 1296us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:34:12.359204 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:34:12.360108 (+ 904us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:34:12.360118 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:34:12.360840 (+ 722us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:34:12.360844 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:34:12.361016 (+ 172us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:34:12.361648 (+ 632us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 18:34:12.361665 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0806 18:34:12.363479 (+ 1814us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 18:34:12.365524 (+ 2045us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:34:12.365529 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:34:12.365543 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:34:12.365905 (+ 362us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:34:12.366243 (+ 338us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:34:12.366246 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:34:12.366248 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:34:12.366387 (+ 139us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:34:12.366770 (+ 383us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 18:34:12.366776 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 18:34:12.366996 (+ 220us) client_negotiation.cc:769] Sending connection context
0806 18:34:12.367198 (+ 202us) client_negotiation.cc:240] Negotiation successful
0806 18:34:12.367499 (+ 301us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":341,"thread_start_us":140,"threads_started":1}
I20260806 18:34:12.369356 21626 heartbeater.cc:347] Connected to a master server at 127.19.194.62:34683
I20260806 18:34:12.369529 21626 heartbeater.cc:464] Registering TS with master...
I20260806 18:34:12.370097 21626 heartbeater.cc:511] Master 127.19.194.62:34683 requested a full tablet report, sending...
I20260806 18:34:12.371713 21406 ts_manager.cc:194] Registered new tserver with Master: ef292515d9c349a8a04b90e9476e31cd (127.19.194.1:39693)
I20260806 18:34:12.372367 20232 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJW69tn/build/debug/bin/kudu as pid 21487
I20260806 18:34:12.372504 20232 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786041246094603-20232-0/minicluster-data/ts-0/wal/instance
I20260806 18:34:12.372758 20232 external_mini_cluster.cc:1428] Setting key e5d5a9578fa698436e7f64ad394596af
I20260806 18:34:12.373139 21406 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.19.194.1@KRBTEST.COM'} at 127.19.194.1:51305
I20260806 18:34:12.379078 21460 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:34:12.371880 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.194.1:47715 (local address 127.19.194.62:34683)
0806 18:34:12.371995 (+ 115us) server_negotiation.cc:206] Beginning negotiation
0806 18:34:12.372000 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 18:34:12.372264 (+ 264us) server_negotiation.cc:407] Connection header received
0806 18:34:12.372402 (+ 138us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:34:12.372405 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:34:12.372448 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:34:12.372517 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:34:12.373040 (+ 523us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:34:12.373726 (+ 686us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:34:12.374697 (+ 971us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:34:12.374925 (+ 228us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:34:12.375029 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:34:12.376085 (+ 1056us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:34:12.376089 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:34:12.376091 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 18:34:12.376113 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:34:12.377507 (+ 1394us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:34:12.377931 (+ 424us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:34:12.377935 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:34:12.377937 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:34:12.378015 (+ 78us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:34:12.378280 (+ 265us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:34:12.378284 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:34:12.378286 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:34:12.378458 (+ 172us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:34:12.378560 (+ 102us) server_negotiation.cc:1035] Waiting for connection context
0806 18:34:12.378813 (+ 253us) server_negotiation.cc:299] Negotiation successful
0806 18:34:12.378916 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":1,"server-negotiator.queue_time_us":49}
I20260806 18:34:12.379094 21632 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:34:12.371754 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.194.62:34683 (local address 127.19.194.1:47715)
0806 18:34:12.372167 (+ 413us) negotiation.cc:107] Waiting for socket to connect
0806 18:34:12.372184 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0806 18:34:12.372296 (+ 112us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:34:12.372614 (+ 318us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:34:12.372617 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:34:12.372806 (+ 189us) client_negotiation.cc:189] Negotiated authn=SASL
0806 18:34:12.372916 (+ 110us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:34:12.372921 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:34:12.373882 (+ 961us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:34:12.373885 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:34:12.374579 (+ 694us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:34:12.374589 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:34:12.375035 (+ 446us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:34:12.375039 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:34:12.375161 (+ 122us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:34:12.375625 (+ 464us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 18:34:12.375638 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0806 18:34:12.375949 (+ 311us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 18:34:12.377633 (+ 1684us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:34:12.377636 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:34:12.377639 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:34:12.377830 (+ 191us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:34:12.378105 (+ 275us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:34:12.378108 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:34:12.378110 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:34:12.378193 (+ 83us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:34:12.378552 (+ 359us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 18:34:12.378555 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 18:34:12.378653 (+ 98us) client_negotiation.cc:769] Sending connection context
0806 18:34:12.378757 (+ 104us) client_negotiation.cc:240] Negotiation successful
0806 18:34:12.378907 (+ 150us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":313,"thread_start_us":134,"threads_started":1}
WARNING: no policy specified for kudu/127.19.194.2@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.19.194.2@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.19.194.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786041246094603-20232-0/krb5kdc/kudu_127.19.194.2.keytab.
Entry for principal kudu/127.19.194.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786041246094603-20232-0/krb5kdc/kudu_127.19.194.2.keytab.
I20260806 18:34:12.401535 20232 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJW69tn/build/debug/bin/kudu
/tmp/dist-test-taskJW69tn/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786041246094603-20232-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786041246094603-20232-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786041246094603-20232-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786041246094603-20232-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.19.194.2:0
--local_ip_for_outbound_sockets=127.19.194.2
--webserver_interface=127.19.194.2
--webserver_port=0
--tserver_master_addrs=127.19.194.62:34683
--keytab_file=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786041246094603-20232-0/krb5kdc/kudu_127.19.194.2.keytab
--principal=kudu/127.19.194.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.19.194.20:40719
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786041246094603-20232-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786041246094603-20232-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786041246094603-20232-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 18:34:12.512463 21635 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 18:34:12.512691 21635 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 18:34:12.512732 21635 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 18:34:12.516669 21635 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 18:34:12.516719 21635 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 18:34:12.516757 21635 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260806 18:34:12.516793 21635 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.19.194.2
I20260806 18:34:12.521809 21635 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.194.20:40719
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786041246094603-20232-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786041246094603-20232-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786041246094603-20232-0/krb5kdc/kudu_127.19.194.2.keytab
--principal=kudu/127.19.194.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.19.194.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786041246094603-20232-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.19.194.2
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.19.194.62:34683
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21635
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.19.194.2
--log_dir=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786041246094603-20232-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 17:59:13 UTC on ccycloud.quasar-wvsnzx.root.comops.site
build id 4571028
I20260806 18:34:12.522666 21635 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 18:34:12.523568 21635 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 18:34:12.532424 21641 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:34:12.532410 21644 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:34:12.532477 21642 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 18:34:12.532548 21635 server_base.cc:1034] running on GCE node
I20260806 18:34:12.533033 21635 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 18:34:12.533653 21635 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 18:34:12.534833 21635 hybrid_clock.cc:648] HybridClock initialized: now 1786041252534815 us; error 31 us; skew 500 ppm
Aug 06 18:34:12 dist-test-slave-1xrh krb5kdc[21356](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041252, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.194.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 18:34:12.537492 21635 init.cc:377] Logged in from keytab as kudu/127.19.194.2@KRBTEST.COM (short username kudu)
I20260806 18:34:12.538321 21635 webserver.cc:460] Webserver started at http://127.19.194.2:36467/ using document root <none> and password file <none>
I20260806 18:34:12.538865 21635 fs_manager.cc:359] Metadata directory not provided
I20260806 18:34:12.538954 21635 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 18:34:12.539233 21635 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 18:34:12.541162 21635 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786041246094603-20232-0/minicluster-data/ts-1/data/instance:
uuid: "4d22b7580880463a9383831743e285e8"
format_stamp: "Formatted at 2026-08-06 18:34:12 on dist-test-slave-1xrh"
server_key: "dfb0d8d809de8ac5b5613449e1250508"
server_key_iv: "4081bb6e4400e9a8dee760a5d7691f81"
server_key_version: "encryptionkey@0"
I20260806 18:34:12.541721 21635 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786041246094603-20232-0/minicluster-data/ts-1/wal/instance:
uuid: "4d22b7580880463a9383831743e285e8"
format_stamp: "Formatted at 2026-08-06 18:34:12 on dist-test-slave-1xrh"
server_key: "dfb0d8d809de8ac5b5613449e1250508"
server_key_iv: "4081bb6e4400e9a8dee760a5d7691f81"
server_key_version: "encryptionkey@0"
I20260806 18:34:12.545353 21635 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.005s sys 0.000s
I20260806 18:34:12.547657 21652 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 18:34:12.548775 21635 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260806 18:34:12.548903 21635 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786041246094603-20232-0/minicluster-data/ts-1/data,/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786041246094603-20232-0/minicluster-data/ts-1/wal
uuid: "4d22b7580880463a9383831743e285e8"
format_stamp: "Formatted at 2026-08-06 18:34:12 on dist-test-slave-1xrh"
server_key: "dfb0d8d809de8ac5b5613449e1250508"
server_key_iv: "4081bb6e4400e9a8dee760a5d7691f81"
server_key_version: "encryptionkey@0"
I20260806 18:34:12.549007 21635 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786041246094603-20232-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786041246094603-20232-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786041246094603-20232-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 18:34:12.566015 21635 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 18:34:12.568656 21635 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 18:34:12.568840 21635 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 18:34:12.570349 21635 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 18:34:12.570418 21635 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 18:34:12.570487 21635 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 18:34:12.570525 21635 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 06 18:34:12 dist-test-slave-1xrh krb5kdc[21356](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041252, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.194.2@KRBTEST.COM for kudu/127.19.194.62@KRBTEST.COM
I20260806 18:34:12.581250 21635 rpc_server.cc:307] RPC server started. Bound to: 127.19.194.2:40521
I20260806 18:34:12.581336 21771 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.194.2:40521 every 8 connection(s)
I20260806 18:34:12.582290 21635 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786041246094603-20232-0/minicluster-data/ts-1/data/info.pb
I20260806 18:34:12.584723 21460 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:34:12.571148 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.194.2:56249 (local address 127.19.194.62:34683)
0806 18:34:12.571291 (+ 143us) server_negotiation.cc:206] Beginning negotiation
0806 18:34:12.571296 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 18:34:12.572639 (+ 1343us) server_negotiation.cc:407] Connection header received
0806 18:34:12.573992 (+ 1353us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:34:12.573995 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:34:12.574043 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:34:12.574119 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:34:12.575813 (+ 1694us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:34:12.576480 (+ 667us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:34:12.578258 (+ 1778us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:34:12.578473 (+ 215us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:34:12.578559 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:34:12.581110 (+ 2551us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:34:12.581116 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:34:12.581120 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 18:34:12.581153 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:34:12.582607 (+ 1454us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:34:12.583227 (+ 620us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:34:12.583230 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:34:12.583231 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:34:12.583306 (+ 75us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:34:12.583660 (+ 354us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:34:12.583663 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:34:12.583665 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:34:12.583857 (+ 192us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:34:12.583965 (+ 108us) server_negotiation.cc:1035] Waiting for connection context
0806 18:34:12.584428 (+ 463us) server_negotiation.cc:299] Negotiation successful
0806 18:34:12.584555 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260806 18:34:12.585417 21661 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:34:12.571975 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.194.62:34683 (local address 127.19.194.2:56249)
0806 18:34:12.572450 (+ 475us) negotiation.cc:107] Waiting for socket to connect
0806 18:34:12.572515 (+ 65us) client_negotiation.cc:174] Beginning negotiation
0806 18:34:12.573762 (+ 1247us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:34:12.574327 (+ 565us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:34:12.574340 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:34:12.574857 (+ 517us) client_negotiation.cc:189] Negotiated authn=SASL
0806 18:34:12.575615 (+ 758us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:34:12.575638 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:34:12.576604 (+ 966us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:34:12.576609 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:34:12.578111 (+ 1502us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:34:12.578121 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:34:12.578661 (+ 540us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:34:12.578666 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:34:12.578865 (+ 199us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:34:12.579482 (+ 617us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 18:34:12.579498 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0806 18:34:12.580902 (+ 1404us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 18:34:12.582723 (+ 1821us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:34:12.582730 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:34:12.582744 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:34:12.583096 (+ 352us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:34:12.583416 (+ 320us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:34:12.583419 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:34:12.583421 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:34:12.583557 (+ 136us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:34:12.583972 (+ 415us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 18:34:12.583980 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 18:34:12.584197 (+ 217us) client_negotiation.cc:769] Sending connection context
0806 18:34:12.584367 (+ 170us) client_negotiation.cc:240] Negotiation successful
0806 18:34:12.584662 (+ 295us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":255,"thread_start_us":109,"threads_started":1}
I20260806 18:34:12.586522 21772 heartbeater.cc:347] Connected to a master server at 127.19.194.62:34683
I20260806 18:34:12.586671 21772 heartbeater.cc:464] Registering TS with master...
I20260806 18:34:12.587182 21772 heartbeater.cc:511] Master 127.19.194.62:34683 requested a full tablet report, sending...
I20260806 18:34:12.587391 20232 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJW69tn/build/debug/bin/kudu as pid 21635
I20260806 18:34:12.587517 20232 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786041246094603-20232-0/minicluster-data/ts-1/wal/instance
I20260806 18:34:12.587838 20232 external_mini_cluster.cc:1428] Setting key f59af2f223f4a0ef9f4b1e63cb0f2f22
I20260806 18:34:12.588281 21405 ts_manager.cc:194] Registered new tserver with Master: 4d22b7580880463a9383831743e285e8 (127.19.194.2:40521)
I20260806 18:34:12.589035 21405 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.19.194.2@KRBTEST.COM'} at 127.19.194.2:56249
WARNING: no policy specified for kudu/127.19.194.3@KRBTEST.COM; defaulting to no policy
I20260806 18:34:12.597330 21460 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:34:12.588793 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.194.2:47805 (local address 127.19.194.62:34683)
0806 18:34:12.589154 (+ 361us) server_negotiation.cc:206] Beginning negotiation
0806 18:34:12.589158 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 18:34:12.589172 (+ 14us) server_negotiation.cc:407] Connection header received
0806 18:34:12.589204 (+ 32us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:34:12.589207 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:34:12.589248 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:34:12.589322 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:34:12.591611 (+ 2289us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:34:12.592105 (+ 494us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:34:12.592920 (+ 815us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:34:12.593138 (+ 218us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:34:12.593239 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:34:12.594310 (+ 1071us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:34:12.594314 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:34:12.594317 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 18:34:12.594339 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:34:12.595725 (+ 1386us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:34:12.596144 (+ 419us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:34:12.596148 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:34:12.596149 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:34:12.596226 (+ 77us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:34:12.596500 (+ 274us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:34:12.596504 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:34:12.596506 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:34:12.596704 (+ 198us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:34:12.596807 (+ 103us) server_negotiation.cc:1035] Waiting for connection context
0806 18:34:12.597073 (+ 266us) server_negotiation.cc:299] Negotiation successful
0806 18:34:12.597180 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":294}
I20260806 18:34:12.597342 21778 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:34:12.588730 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.194.62:34683 (local address 127.19.194.2:47805)
0806 18:34:12.589016 (+ 286us) negotiation.cc:107] Waiting for socket to connect
0806 18:34:12.589032 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0806 18:34:12.589121 (+ 89us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:34:12.591232 (+ 2111us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:34:12.591235 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:34:12.591405 (+ 170us) client_negotiation.cc:189] Negotiated authn=SASL
0806 18:34:12.591498 (+ 93us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:34:12.591504 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:34:12.592225 (+ 721us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:34:12.592228 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:34:12.592804 (+ 576us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:34:12.592811 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:34:12.593257 (+ 446us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:34:12.593261 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:34:12.593354 (+ 93us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:34:12.593847 (+ 493us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 18:34:12.593861 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0806 18:34:12.594176 (+ 315us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 18:34:12.595839 (+ 1663us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:34:12.595844 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:34:12.595846 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:34:12.596047 (+ 201us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:34:12.596317 (+ 270us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:34:12.596320 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:34:12.596322 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:34:12.596407 (+ 85us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:34:12.596812 (+ 405us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 18:34:12.596816 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 18:34:12.596907 (+ 91us) client_negotiation.cc:769] Sending connection context
0806 18:34:12.597016 (+ 109us) client_negotiation.cc:240] Negotiation successful
0806 18:34:12.597156 (+ 140us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":202,"thread_start_us":119,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.19.194.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.19.194.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786041246094603-20232-0/krb5kdc/kudu_127.19.194.3.keytab.
Entry for principal kudu/127.19.194.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786041246094603-20232-0/krb5kdc/kudu_127.19.194.3.keytab.
I20260806 18:34:12.614208 20232 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJW69tn/build/debug/bin/kudu
/tmp/dist-test-taskJW69tn/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786041246094603-20232-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786041246094603-20232-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786041246094603-20232-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786041246094603-20232-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.19.194.3:0
--local_ip_for_outbound_sockets=127.19.194.3
--webserver_interface=127.19.194.3
--webserver_port=0
--tserver_master_addrs=127.19.194.62:34683
--keytab_file=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786041246094603-20232-0/krb5kdc/kudu_127.19.194.3.keytab
--principal=kudu/127.19.194.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.19.194.20:40719
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786041246094603-20232-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786041246094603-20232-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786041246094603-20232-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 18:34:12.726428 21781 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 18:34:12.726681 21781 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 18:34:12.726724 21781 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 18:34:12.730532 21781 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 18:34:12.730580 21781 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 18:34:12.730615 21781 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260806 18:34:12.730645 21781 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.19.194.3
I20260806 18:34:12.735597 21781 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.194.20:40719
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786041246094603-20232-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786041246094603-20232-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786041246094603-20232-0/krb5kdc/kudu_127.19.194.3.keytab
--principal=kudu/127.19.194.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.19.194.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786041246094603-20232-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.19.194.3
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.19.194.62:34683
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21781
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.19.194.3
--log_dir=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786041246094603-20232-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 17:59:13 UTC on ccycloud.quasar-wvsnzx.root.comops.site
build id 4571028
I20260806 18:34:12.736526 21781 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 18:34:12.737438 21781 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 18:34:12.746870 21788 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:34:12.746866 21790 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:34:12.746867 21787 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 18:34:12.747211 21781 server_base.cc:1034] running on GCE node
I20260806 18:34:12.747592 21781 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 18:34:12.748183 21781 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 18:34:12.749343 21781 hybrid_clock.cc:648] HybridClock initialized: now 1786041252749324 us; error 30 us; skew 500 ppm
Aug 06 18:34:12 dist-test-slave-1xrh krb5kdc[21356](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041252, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.194.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 18:34:12.752233 21781 init.cc:377] Logged in from keytab as kudu/127.19.194.3@KRBTEST.COM (short username kudu)
I20260806 18:34:12.753165 21781 webserver.cc:460] Webserver started at http://127.19.194.3:45143/ using document root <none> and password file <none>
I20260806 18:34:12.753707 21781 fs_manager.cc:359] Metadata directory not provided
I20260806 18:34:12.753803 21781 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 18:34:12.754014 21781 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 18:34:12.756050 21781 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786041246094603-20232-0/minicluster-data/ts-2/data/instance:
uuid: "8b1a57b1c16a4f71a055f46154de9947"
format_stamp: "Formatted at 2026-08-06 18:34:12 on dist-test-slave-1xrh"
server_key: "ee0e312e61148edcb268147615ca0e30"
server_key_iv: "db6c65cd4d8466dfce3742f07ee3d447"
server_key_version: "encryptionkey@0"
I20260806 18:34:12.756599 21781 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786041246094603-20232-0/minicluster-data/ts-2/wal/instance:
uuid: "8b1a57b1c16a4f71a055f46154de9947"
format_stamp: "Formatted at 2026-08-06 18:34:12 on dist-test-slave-1xrh"
server_key: "ee0e312e61148edcb268147615ca0e30"
server_key_iv: "db6c65cd4d8466dfce3742f07ee3d447"
server_key_version: "encryptionkey@0"
I20260806 18:34:12.760231 21781 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.001s sys 0.003s
I20260806 18:34:12.762568 21798 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 18:34:12.763746 21781 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260806 18:34:12.763916 21781 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786041246094603-20232-0/minicluster-data/ts-2/data,/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786041246094603-20232-0/minicluster-data/ts-2/wal
uuid: "8b1a57b1c16a4f71a055f46154de9947"
format_stamp: "Formatted at 2026-08-06 18:34:12 on dist-test-slave-1xrh"
server_key: "ee0e312e61148edcb268147615ca0e30"
server_key_iv: "db6c65cd4d8466dfce3742f07ee3d447"
server_key_version: "encryptionkey@0"
I20260806 18:34:12.764029 21781 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786041246094603-20232-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786041246094603-20232-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786041246094603-20232-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 18:34:12.786576 21781 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 18:34:12.789505 21781 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 18:34:12.789654 21781 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 18:34:12.790989 21781 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 18:34:12.791046 21781 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 18:34:12.791109 21781 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 18:34:12.791167 21781 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 06 18:34:12 dist-test-slave-1xrh krb5kdc[21356](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041252, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.194.3@KRBTEST.COM for kudu/127.19.194.62@KRBTEST.COM
I20260806 18:34:12.802448 21781 rpc_server.cc:307] RPC server started. Bound to: 127.19.194.3:34037
I20260806 18:34:12.802610 21913 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.194.3:34037 every 8 connection(s)
I20260806 18:34:12.803385 21781 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786041246094603-20232-0/minicluster-data/ts-2/data/info.pb
I20260806 18:34:12.805496 21460 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:34:12.791886 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.194.3:53393 (local address 127.19.194.62:34683)
0806 18:34:12.792011 (+ 125us) server_negotiation.cc:206] Beginning negotiation
0806 18:34:12.792014 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 18:34:12.793077 (+ 1063us) server_negotiation.cc:407] Connection header received
0806 18:34:12.794013 (+ 936us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:34:12.794020 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:34:12.794090 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:34:12.794193 (+ 103us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:34:12.795947 (+ 1754us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:34:12.796606 (+ 659us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:34:12.798446 (+ 1840us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:34:12.798825 (+ 379us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:34:12.798952 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:34:12.801794 (+ 2842us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:34:12.801800 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:34:12.801806 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 18:34:12.801843 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:34:12.803614 (+ 1771us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:34:12.804234 (+ 620us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:34:12.804237 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:34:12.804238 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:34:12.804318 (+ 80us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:34:12.804623 (+ 305us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:34:12.804626 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:34:12.804627 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:34:12.804790 (+ 163us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:34:12.804866 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0806 18:34:12.805287 (+ 421us) server_negotiation.cc:299] Negotiation successful
0806 18:34:12.805380 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260806 18:34:12.806267 21807 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:34:12.792430 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.194.62:34683 (local address 127.19.194.3:53393)
0806 18:34:12.792931 (+ 501us) negotiation.cc:107] Waiting for socket to connect
0806 18:34:12.792978 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0806 18:34:12.793771 (+ 793us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:34:12.794519 (+ 748us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:34:12.794541 (+ 22us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:34:12.795041 (+ 500us) client_negotiation.cc:189] Negotiated authn=SASL
0806 18:34:12.795763 (+ 722us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:34:12.795787 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:34:12.796739 (+ 952us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:34:12.796744 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:34:12.798274 (+ 1530us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:34:12.798286 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:34:12.799020 (+ 734us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:34:12.799024 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:34:12.799269 (+ 245us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:34:12.799814 (+ 545us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 18:34:12.799836 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0806 18:34:12.801620 (+ 1784us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 18:34:12.803730 (+ 2110us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:34:12.803736 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:34:12.803750 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:34:12.804136 (+ 386us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:34:12.804414 (+ 278us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:34:12.804417 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:34:12.804418 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:34:12.804543 (+ 125us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:34:12.804878 (+ 335us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 18:34:12.804882 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 18:34:12.805088 (+ 206us) client_negotiation.cc:769] Sending connection context
0806 18:34:12.805242 (+ 154us) client_negotiation.cc:240] Negotiation successful
0806 18:34:12.805459 (+ 217us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":324,"thread_start_us":138,"threads_started":1}
I20260806 18:34:12.807430 21914 heartbeater.cc:347] Connected to a master server at 127.19.194.62:34683
I20260806 18:34:12.807569 21914 heartbeater.cc:464] Registering TS with master...
I20260806 18:34:12.808018 21914 heartbeater.cc:511] Master 127.19.194.62:34683 requested a full tablet report, sending...
I20260806 18:34:12.809052 21406 ts_manager.cc:194] Registered new tserver with Master: 8b1a57b1c16a4f71a055f46154de9947 (127.19.194.3:34037)
I20260806 18:34:12.809727 21406 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.19.194.3@KRBTEST.COM'} at 127.19.194.3:53393
I20260806 18:34:12.809866 20232 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJW69tn/build/debug/bin/kudu as pid 21781
I20260806 18:34:12.809978 20232 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786041246094603-20232-0/minicluster-data/ts-2/wal/instance
I20260806 18:34:12.810220 20232 external_mini_cluster.cc:1428] Setting key c4241b044b3ea4f698423e5c3fe0241a
I20260806 18:34:12.812224 20232 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260806 18:34:12.817615 21920 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:34:12.809490 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.194.62:34683 (local address 127.19.194.3:43701)
0806 18:34:12.810700 (+ 1210us) negotiation.cc:107] Waiting for socket to connect
0806 18:34:12.810712 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0806 18:34:12.810821 (+ 109us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:34:12.811072 (+ 251us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:34:12.811075 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:34:12.811295 (+ 220us) client_negotiation.cc:189] Negotiated authn=SASL
0806 18:34:12.811404 (+ 109us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:34:12.811411 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:34:12.812389 (+ 978us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:34:12.812393 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:34:12.812927 (+ 534us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:34:12.812935 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:34:12.813578 (+ 643us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:34:12.813581 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:34:12.813654 (+ 73us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:34:12.814119 (+ 465us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 18:34:12.814131 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0806 18:34:12.814405 (+ 274us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 18:34:12.816157 (+ 1752us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:34:12.816160 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:34:12.816162 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:34:12.816355 (+ 193us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:34:12.816654 (+ 299us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:34:12.816657 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:34:12.816659 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:34:12.816753 (+ 94us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:34:12.817242 (+ 489us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 18:34:12.817244 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 18:34:12.817316 (+ 72us) client_negotiation.cc:769] Sending connection context
0806 18:34:12.817395 (+ 79us) client_negotiation.cc:240] Negotiation successful
0806 18:34:12.817493 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1126,"thread_start_us":121,"threads_started":1}
I20260806 18:34:12.817812 21460 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:34:12.809547 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.194.3:43701 (local address 127.19.194.62:34683)
0806 18:34:12.809709 (+ 162us) server_negotiation.cc:206] Beginning negotiation
0806 18:34:12.809716 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 18:34:12.810782 (+ 1066us) server_negotiation.cc:407] Connection header received
0806 18:34:12.810908 (+ 126us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:34:12.810910 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:34:12.810968 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:34:12.811055 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:34:12.811518 (+ 463us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:34:12.812176 (+ 658us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:34:12.813053 (+ 877us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:34:12.813434 (+ 381us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:34:12.813555 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:34:12.814561 (+ 1006us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:34:12.814565 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:34:12.814567 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 18:34:12.814591 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:34:12.816036 (+ 1445us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:34:12.816454 (+ 418us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:34:12.816458 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:34:12.816467 (+ 9us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:34:12.816547 (+ 80us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:34:12.816876 (+ 329us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:34:12.816883 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:34:12.816887 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:34:12.817144 (+ 257us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:34:12.817257 (+ 113us) server_negotiation.cc:1035] Waiting for connection context
0806 18:34:12.817487 (+ 230us) server_negotiation.cc:299] Negotiation successful
0806 18:34:12.817601 (+ 114us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
Aug 06 18:34:12 dist-test-slave-1xrh krb5kdc[21356](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041252, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 06 18:34:12 dist-test-slave-1xrh krb5kdc[21356](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041252, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.19.194.62@KRBTEST.COM
I20260806 18:34:12.837818 21460 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:34:12.830570 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47880 (local address 127.19.194.62:34683)
0806 18:34:12.830664 (+ 94us) server_negotiation.cc:206] Beginning negotiation
0806 18:34:12.830667 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 18:34:12.830676 (+ 9us) server_negotiation.cc:407] Connection header received
0806 18:34:12.830702 (+ 26us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:34:12.830704 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:34:12.830734 (+ 30us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:34:12.830791 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:34:12.831516 (+ 725us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:34:12.831986 (+ 470us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:34:12.832770 (+ 784us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:34:12.832985 (+ 215us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:34:12.833070 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:34:12.835278 (+ 2208us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:34:12.835281 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:34:12.835283 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 18:34:12.835304 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:34:12.836456 (+ 1152us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:34:12.836818 (+ 362us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:34:12.836820 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:34:12.836821 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:34:12.836890 (+ 69us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:34:12.837115 (+ 225us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:34:12.837117 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:34:12.837118 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:34:12.837262 (+ 144us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:34:12.837354 (+ 92us) server_negotiation.cc:1035] Waiting for connection context
0806 18:34:12.837542 (+ 188us) server_negotiation.cc:299] Negotiation successful
0806 18:34:12.837624 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":30}
I20260806 18:34:12.840435 21406 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:47880:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260806 18:34:12.843462 21406 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 18:34:12.857193 21933 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:34:12.851258 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46366 (local address 127.19.194.3:34037)
0806 18:34:12.852291 (+ 1033us) server_negotiation.cc:206] Beginning negotiation
0806 18:34:12.852306 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0806 18:34:12.852331 (+ 25us) server_negotiation.cc:407] Connection header received
0806 18:34:12.852407 (+ 76us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:34:12.852413 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:34:12.852623 (+ 210us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:34:12.852744 (+ 121us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 18:34:12.853048 (+ 304us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:34:12.854128 (+ 1080us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:34:12.855755 (+ 1627us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:34:12.856489 (+ 734us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:34:12.856595 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:34:12.856744 (+ 149us) server_negotiation.cc:1035] Waiting for connection context
0806 18:34:12.856940 (+ 196us) server_negotiation.cc:299] Negotiation successful
0806 18:34:12.857056 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":940,"thread_start_us":130,"threads_started":1}
I20260806 18:34:12.857182 21932 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:34:12.851111 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.194.3:34037 (local address 127.0.0.1:46366)
0806 18:34:12.851608 (+ 497us) negotiation.cc:107] Waiting for socket to connect
0806 18:34:12.851658 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0806 18:34:12.851785 (+ 127us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:34:12.852753 (+ 968us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:34:12.852761 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:34:12.852784 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 18:34:12.852920 (+ 136us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:34:12.852927 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:34:12.854247 (+ 1320us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:34:12.854250 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:34:12.855625 (+ 1375us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:34:12.855636 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:34:12.856621 (+ 985us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:34:12.856625 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:34:12.856745 (+ 120us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:34:12.856785 (+ 40us) client_negotiation.cc:769] Sending connection context
0806 18:34:12.856895 (+ 110us) client_negotiation.cc:240] Negotiation successful
0806 18:34:12.857011 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":325,"thread_start_us":68,"threads_started":1}
I20260806 18:34:12.857415 21935 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:34:12.851849 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.194.2:40521 (local address 127.0.0.1:47116)
0806 18:34:12.852838 (+ 989us) negotiation.cc:107] Waiting for socket to connect
0806 18:34:12.852850 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0806 18:34:12.852937 (+ 87us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:34:12.853360 (+ 423us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:34:12.853363 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:34:12.853382 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 18:34:12.853598 (+ 216us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:34:12.853606 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:34:12.854812 (+ 1206us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:34:12.854815 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:34:12.855805 (+ 990us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:34:12.855816 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:34:12.856951 (+ 1135us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:34:12.856955 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:34:12.857030 (+ 75us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:34:12.857050 (+ 20us) client_negotiation.cc:769] Sending connection context
0806 18:34:12.857118 (+ 68us) client_negotiation.cc:240] Negotiation successful
0806 18:34:12.857176 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":933,"thread_start_us":72,"threads_started":1}
I20260806 18:34:12.857654 21936 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:34:12.851748 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.194.1:39693 (local address 127.0.0.1:55038)
0806 18:34:12.852533 (+ 785us) negotiation.cc:107] Waiting for socket to connect
0806 18:34:12.852545 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0806 18:34:12.852654 (+ 109us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:34:12.853519 (+ 865us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:34:12.853521 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:34:12.853537 (+ 16us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 18:34:12.853616 (+ 79us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:34:12.853621 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:34:12.854981 (+ 1360us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:34:12.854984 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:34:12.856109 (+ 1125us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:34:12.856119 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:34:12.857099 (+ 980us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:34:12.857103 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:34:12.857195 (+ 92us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:34:12.857228 (+ 33us) client_negotiation.cc:769] Sending connection context
0806 18:34:12.857455 (+ 227us) client_negotiation.cc:240] Negotiation successful
0806 18:34:12.857539 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":709,"thread_start_us":102,"threads_started":1}
I20260806 18:34:12.857702 21937 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:34:12.852163 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47116 (local address 127.19.194.2:40521)
0806 18:34:12.852457 (+ 294us) server_negotiation.cc:206] Beginning negotiation
0806 18:34:12.852469 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0806 18:34:12.852940 (+ 471us) server_negotiation.cc:407] Connection header received
0806 18:34:12.853038 (+ 98us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:34:12.853043 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:34:12.853224 (+ 181us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:34:12.853376 (+ 152us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 18:34:12.853720 (+ 344us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:34:12.854686 (+ 966us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:34:12.855935 (+ 1249us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:34:12.856642 (+ 707us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:34:12.856745 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:34:12.856894 (+ 149us) server_negotiation.cc:1035] Waiting for connection context
0806 18:34:12.857252 (+ 358us) server_negotiation.cc:299] Negotiation successful
0806 18:34:12.857558 (+ 306us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":193,"thread_start_us":109,"threads_started":1}
I20260806 18:34:12.858295 21934 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:34:12.851417 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55038 (local address 127.19.194.1:39693)
0806 18:34:12.852096 (+ 679us) server_negotiation.cc:206] Beginning negotiation
0806 18:34:12.852114 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0806 18:34:12.852638 (+ 524us) server_negotiation.cc:407] Connection header received
0806 18:34:12.853136 (+ 498us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:34:12.853141 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:34:12.853366 (+ 225us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:34:12.853758 (+ 392us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 18:34:12.853881 (+ 123us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:34:12.854847 (+ 966us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:34:12.856233 (+ 1386us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:34:12.856962 (+ 729us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:34:12.857845 (+ 883us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:34:12.857998 (+ 153us) server_negotiation.cc:1035] Waiting for connection context
0806 18:34:12.858048 (+ 50us) server_negotiation.cc:299] Negotiation successful
0806 18:34:12.858137 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":556,"thread_start_us":90,"threads_started":1}
I20260806 18:34:12.860322 21848 tablet_service.cc:1467] Processing CreateTablet for tablet 098ba0adf30c47d49848f5428d7f3cc5 (DEFAULT_TABLE table=test-table [id=d6bdfac9aa3849beb05bfb11bc2f1fd5]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 18:34:12.860322 21702 tablet_service.cc:1467] Processing CreateTablet for tablet 098ba0adf30c47d49848f5428d7f3cc5 (DEFAULT_TABLE table=test-table [id=d6bdfac9aa3849beb05bfb11bc2f1fd5]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 18:34:12.860742 21556 tablet_service.cc:1467] Processing CreateTablet for tablet 098ba0adf30c47d49848f5428d7f3cc5 (DEFAULT_TABLE table=test-table [id=d6bdfac9aa3849beb05bfb11bc2f1fd5]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 18:34:12.861464 21848 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 098ba0adf30c47d49848f5428d7f3cc5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 18:34:12.861459 21702 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 098ba0adf30c47d49848f5428d7f3cc5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 18:34:12.861713 21556 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 098ba0adf30c47d49848f5428d7f3cc5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 18:34:12.868678 21938 tablet_bootstrap.cc:492] T 098ba0adf30c47d49848f5428d7f3cc5 P ef292515d9c349a8a04b90e9476e31cd: Bootstrap starting.
I20260806 18:34:12.869918 21940 tablet_bootstrap.cc:492] T 098ba0adf30c47d49848f5428d7f3cc5 P 4d22b7580880463a9383831743e285e8: Bootstrap starting.
I20260806 18:34:12.870180 21939 tablet_bootstrap.cc:492] T 098ba0adf30c47d49848f5428d7f3cc5 P 8b1a57b1c16a4f71a055f46154de9947: Bootstrap starting.
I20260806 18:34:12.870952 21938 tablet_bootstrap.cc:654] T 098ba0adf30c47d49848f5428d7f3cc5 P ef292515d9c349a8a04b90e9476e31cd: Neither blocks nor log segments found. Creating new log.
I20260806 18:34:12.871722 21938 log.cc:824] T 098ba0adf30c47d49848f5428d7f3cc5 P ef292515d9c349a8a04b90e9476e31cd: Log is configured to *not* fsync() on all Append() calls
I20260806 18:34:12.872071 21940 tablet_bootstrap.cc:654] T 098ba0adf30c47d49848f5428d7f3cc5 P 4d22b7580880463a9383831743e285e8: Neither blocks nor log segments found. Creating new log.
I20260806 18:34:12.872074 21939 tablet_bootstrap.cc:654] T 098ba0adf30c47d49848f5428d7f3cc5 P 8b1a57b1c16a4f71a055f46154de9947: Neither blocks nor log segments found. Creating new log.
I20260806 18:34:12.872905 21939 log.cc:824] T 098ba0adf30c47d49848f5428d7f3cc5 P 8b1a57b1c16a4f71a055f46154de9947: Log is configured to *not* fsync() on all Append() calls
I20260806 18:34:12.872905 21940 log.cc:824] T 098ba0adf30c47d49848f5428d7f3cc5 P 4d22b7580880463a9383831743e285e8: Log is configured to *not* fsync() on all Append() calls
I20260806 18:34:12.873809 21938 tablet_bootstrap.cc:492] T 098ba0adf30c47d49848f5428d7f3cc5 P ef292515d9c349a8a04b90e9476e31cd: No bootstrap required, opened a new log
I20260806 18:34:12.873997 21938 ts_tablet_manager.cc:1397] T 098ba0adf30c47d49848f5428d7f3cc5 P ef292515d9c349a8a04b90e9476e31cd: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260806 18:34:12.874802 21939 tablet_bootstrap.cc:492] T 098ba0adf30c47d49848f5428d7f3cc5 P 8b1a57b1c16a4f71a055f46154de9947: No bootstrap required, opened a new log
I20260806 18:34:12.874802 21940 tablet_bootstrap.cc:492] T 098ba0adf30c47d49848f5428d7f3cc5 P 4d22b7580880463a9383831743e285e8: No bootstrap required, opened a new log
I20260806 18:34:12.874980 21939 ts_tablet_manager.cc:1397] T 098ba0adf30c47d49848f5428d7f3cc5 P 8b1a57b1c16a4f71a055f46154de9947: Time spent bootstrapping tablet: real 0.005s user 0.003s sys 0.000s
I20260806 18:34:12.874980 21940 ts_tablet_manager.cc:1397] T 098ba0adf30c47d49848f5428d7f3cc5 P 4d22b7580880463a9383831743e285e8: Time spent bootstrapping tablet: real 0.005s user 0.004s sys 0.000s
I20260806 18:34:12.876890 21938 raft_consensus.cc:348] T 098ba0adf30c47d49848f5428d7f3cc5 P ef292515d9c349a8a04b90e9476e31cd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b1a57b1c16a4f71a055f46154de9947" member_type: VOTER last_known_addr { host: "127.19.194.3" port: 34037 } } peers { permanent_uuid: "ef292515d9c349a8a04b90e9476e31cd" member_type: VOTER last_known_addr { host: "127.19.194.1" port: 39693 } } peers { permanent_uuid: "4d22b7580880463a9383831743e285e8" member_type: VOTER last_known_addr { host: "127.19.194.2" port: 40521 } }
I20260806 18:34:12.877131 21938 raft_consensus.cc:374] T 098ba0adf30c47d49848f5428d7f3cc5 P ef292515d9c349a8a04b90e9476e31cd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 18:34:12.877185 21938 raft_consensus.cc:729] T 098ba0adf30c47d49848f5428d7f3cc5 P ef292515d9c349a8a04b90e9476e31cd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ef292515d9c349a8a04b90e9476e31cd, State: Initialized, Role: FOLLOWER
I20260806 18:34:12.877606 21938 consensus_queue.cc:260] T 098ba0adf30c47d49848f5428d7f3cc5 P ef292515d9c349a8a04b90e9476e31cd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b1a57b1c16a4f71a055f46154de9947" member_type: VOTER last_known_addr { host: "127.19.194.3" port: 34037 } } peers { permanent_uuid: "ef292515d9c349a8a04b90e9476e31cd" member_type: VOTER last_known_addr { host: "127.19.194.1" port: 39693 } } peers { permanent_uuid: "4d22b7580880463a9383831743e285e8" member_type: VOTER last_known_addr { host: "127.19.194.2" port: 40521 } }
I20260806 18:34:12.878266 21626 heartbeater.cc:503] Master 127.19.194.62:34683 was elected leader, sending a full tablet report...
I20260806 18:34:12.878556 21938 ts_tablet_manager.cc:1428] T 098ba0adf30c47d49848f5428d7f3cc5 P ef292515d9c349a8a04b90e9476e31cd: Time spent starting tablet: real 0.004s user 0.005s sys 0.000s
I20260806 18:34:12.878839 21940 raft_consensus.cc:348] T 098ba0adf30c47d49848f5428d7f3cc5 P 4d22b7580880463a9383831743e285e8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b1a57b1c16a4f71a055f46154de9947" member_type: VOTER last_known_addr { host: "127.19.194.3" port: 34037 } } peers { permanent_uuid: "ef292515d9c349a8a04b90e9476e31cd" member_type: VOTER last_known_addr { host: "127.19.194.1" port: 39693 } } peers { permanent_uuid: "4d22b7580880463a9383831743e285e8" member_type: VOTER last_known_addr { host: "127.19.194.2" port: 40521 } }
I20260806 18:34:12.879073 21940 raft_consensus.cc:374] T 098ba0adf30c47d49848f5428d7f3cc5 P 4d22b7580880463a9383831743e285e8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 18:34:12.879154 21940 raft_consensus.cc:729] T 098ba0adf30c47d49848f5428d7f3cc5 P 4d22b7580880463a9383831743e285e8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4d22b7580880463a9383831743e285e8, State: Initialized, Role: FOLLOWER
I20260806 18:34:12.879571 21940 consensus_queue.cc:260] T 098ba0adf30c47d49848f5428d7f3cc5 P 4d22b7580880463a9383831743e285e8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b1a57b1c16a4f71a055f46154de9947" member_type: VOTER last_known_addr { host: "127.19.194.3" port: 34037 } } peers { permanent_uuid: "ef292515d9c349a8a04b90e9476e31cd" member_type: VOTER last_known_addr { host: "127.19.194.1" port: 39693 } } peers { permanent_uuid: "4d22b7580880463a9383831743e285e8" member_type: VOTER last_known_addr { host: "127.19.194.2" port: 40521 } }
I20260806 18:34:12.880033 21939 raft_consensus.cc:348] T 098ba0adf30c47d49848f5428d7f3cc5 P 8b1a57b1c16a4f71a055f46154de9947 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b1a57b1c16a4f71a055f46154de9947" member_type: VOTER last_known_addr { host: "127.19.194.3" port: 34037 } } peers { permanent_uuid: "ef292515d9c349a8a04b90e9476e31cd" member_type: VOTER last_known_addr { host: "127.19.194.1" port: 39693 } } peers { permanent_uuid: "4d22b7580880463a9383831743e285e8" member_type: VOTER last_known_addr { host: "127.19.194.2" port: 40521 } }
I20260806 18:34:12.880262 21939 raft_consensus.cc:374] T 098ba0adf30c47d49848f5428d7f3cc5 P 8b1a57b1c16a4f71a055f46154de9947 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 18:34:12.880316 21939 raft_consensus.cc:729] T 098ba0adf30c47d49848f5428d7f3cc5 P 8b1a57b1c16a4f71a055f46154de9947 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8b1a57b1c16a4f71a055f46154de9947, State: Initialized, Role: FOLLOWER
I20260806 18:34:12.880508 21940 ts_tablet_manager.cc:1428] T 098ba0adf30c47d49848f5428d7f3cc5 P 4d22b7580880463a9383831743e285e8: Time spent starting tablet: real 0.005s user 0.003s sys 0.003s
I20260806 18:34:12.880735 21939 consensus_queue.cc:260] T 098ba0adf30c47d49848f5428d7f3cc5 P 8b1a57b1c16a4f71a055f46154de9947 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b1a57b1c16a4f71a055f46154de9947" member_type: VOTER last_known_addr { host: "127.19.194.3" port: 34037 } } peers { permanent_uuid: "ef292515d9c349a8a04b90e9476e31cd" member_type: VOTER last_known_addr { host: "127.19.194.1" port: 39693 } } peers { permanent_uuid: "4d22b7580880463a9383831743e285e8" member_type: VOTER last_known_addr { host: "127.19.194.2" port: 40521 } }
I20260806 18:34:12.880846 21772 heartbeater.cc:503] Master 127.19.194.62:34683 was elected leader, sending a full tablet report...
I20260806 18:34:12.881373 21914 heartbeater.cc:503] Master 127.19.194.62:34683 was elected leader, sending a full tablet report...
I20260806 18:34:12.881688 21939 ts_tablet_manager.cc:1428] T 098ba0adf30c47d49848f5428d7f3cc5 P 8b1a57b1c16a4f71a055f46154de9947: Time spent starting tablet: real 0.007s user 0.001s sys 0.005s
I20260806 18:34:12.888854 21946 raft_consensus.cc:482] T 098ba0adf30c47d49848f5428d7f3cc5 P 8b1a57b1c16a4f71a055f46154de9947 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 18:34:12.888990 21946 raft_consensus.cc:504] T 098ba0adf30c47d49848f5428d7f3cc5 P 8b1a57b1c16a4f71a055f46154de9947 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b1a57b1c16a4f71a055f46154de9947" member_type: VOTER last_known_addr { host: "127.19.194.3" port: 34037 } } peers { permanent_uuid: "ef292515d9c349a8a04b90e9476e31cd" member_type: VOTER last_known_addr { host: "127.19.194.1" port: 39693 } } peers { permanent_uuid: "4d22b7580880463a9383831743e285e8" member_type: VOTER last_known_addr { host: "127.19.194.2" port: 40521 } }
I20260806 18:34:12.889873 21946 leader_election.cc:288] T 098ba0adf30c47d49848f5428d7f3cc5 P 8b1a57b1c16a4f71a055f46154de9947 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ef292515d9c349a8a04b90e9476e31cd (127.19.194.1:39693), 4d22b7580880463a9383831743e285e8 (127.19.194.2:40521)
I20260806 18:34:12.894456 21937 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:34:12.890423 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.194.3:57089 (local address 127.19.194.2:40521)
0806 18:34:12.890555 (+ 132us) server_negotiation.cc:206] Beginning negotiation
0806 18:34:12.890559 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 18:34:12.890576 (+ 17us) server_negotiation.cc:407] Connection header received
0806 18:34:12.890611 (+ 35us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:34:12.890613 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:34:12.890644 (+ 31us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:34:12.890701 (+ 57us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 18:34:12.890986 (+ 285us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:34:12.891746 (+ 760us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:34:12.893361 (+ 1615us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:34:12.893911 (+ 550us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:34:12.894035 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:34:12.894088 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0806 18:34:12.894267 (+ 179us) server_negotiation.cc:299] Negotiation successful
0806 18:34:12.894349 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260806 18:34:12.894477 21948 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:34:12.890084 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.194.2:40521 (local address 127.19.194.3:57089)
0806 18:34:12.890347 (+ 263us) negotiation.cc:107] Waiting for socket to connect
0806 18:34:12.890363 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0806 18:34:12.890456 (+ 93us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:34:12.890734 (+ 278us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:34:12.890736 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:34:12.890750 (+ 14us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 18:34:12.890875 (+ 125us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:34:12.890880 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:34:12.891838 (+ 958us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:34:12.891841 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:34:12.893241 (+ 1400us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:34:12.893254 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:34:12.894030 (+ 776us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:34:12.894033 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:34:12.894119 (+ 86us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:34:12.894141 (+ 22us) client_negotiation.cc:769] Sending connection context
0806 18:34:12.894238 (+ 97us) client_negotiation.cc:240] Negotiation successful
0806 18:34:12.894314 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":188,"mutex_wait_us":21,"thread_start_us":82,"threads_started":1}
I20260806 18:34:12.894677 21934 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:34:12.890087 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.194.3:59865 (local address 127.19.194.1:39693)
0806 18:34:12.890216 (+ 129us) server_negotiation.cc:206] Beginning negotiation
0806 18:34:12.890221 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 18:34:12.890294 (+ 73us) server_negotiation.cc:407] Connection header received
0806 18:34:12.890450 (+ 156us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:34:12.890453 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:34:12.890506 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:34:12.890575 (+ 69us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 18:34:12.890855 (+ 280us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:34:12.891680 (+ 825us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:34:12.893557 (+ 1877us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:34:12.894128 (+ 571us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:34:12.894252 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:34:12.894297 (+ 45us) server_negotiation.cc:1035] Waiting for connection context
0806 18:34:12.894495 (+ 198us) server_negotiation.cc:299] Negotiation successful
0806 18:34:12.894569 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260806 18:34:12.894691 21807 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:34:12.890050 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.194.1:39693 (local address 127.19.194.3:59865)
0806 18:34:12.890237 (+ 187us) negotiation.cc:107] Waiting for socket to connect
0806 18:34:12.890251 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0806 18:34:12.890337 (+ 86us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:34:12.890592 (+ 255us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:34:12.890594 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:34:12.890608 (+ 14us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 18:34:12.890744 (+ 136us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:34:12.890751 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:34:12.891780 (+ 1029us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:34:12.891783 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:34:12.893452 (+ 1669us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:34:12.893460 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:34:12.894256 (+ 796us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:34:12.894260 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:34:12.894363 (+ 103us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:34:12.894384 (+ 21us) client_negotiation.cc:769] Sending connection context
0806 18:34:12.894458 (+ 74us) client_negotiation.cc:240] Negotiation successful
0806 18:34:12.894543 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":119,"mutex_wait_us":38}
I20260806 18:34:12.895006 21722 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "098ba0adf30c47d49848f5428d7f3cc5" candidate_uuid: "8b1a57b1c16a4f71a055f46154de9947" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4d22b7580880463a9383831743e285e8" is_pre_election: true
I20260806 18:34:12.895273 21578 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "098ba0adf30c47d49848f5428d7f3cc5" candidate_uuid: "8b1a57b1c16a4f71a055f46154de9947" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ef292515d9c349a8a04b90e9476e31cd" is_pre_election: true
I20260806 18:34:12.895308 21722 raft_consensus.cc:2454] T 098ba0adf30c47d49848f5428d7f3cc5 P 4d22b7580880463a9383831743e285e8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8b1a57b1c16a4f71a055f46154de9947 in term 0.
I20260806 18:34:12.895507 21578 raft_consensus.cc:2454] T 098ba0adf30c47d49848f5428d7f3cc5 P ef292515d9c349a8a04b90e9476e31cd [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8b1a57b1c16a4f71a055f46154de9947 in term 0.
I20260806 18:34:12.895768 21801 leader_election.cc:302] T 098ba0adf30c47d49848f5428d7f3cc5 P 8b1a57b1c16a4f71a055f46154de9947 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4d22b7580880463a9383831743e285e8, 8b1a57b1c16a4f71a055f46154de9947; no voters:
I20260806 18:34:12.896030 21946 raft_consensus.cc:2781] T 098ba0adf30c47d49848f5428d7f3cc5 P 8b1a57b1c16a4f71a055f46154de9947 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 18:34:12.896111 21946 raft_consensus.cc:482] T 098ba0adf30c47d49848f5428d7f3cc5 P 8b1a57b1c16a4f71a055f46154de9947 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 18:34:12.896166 21946 raft_consensus.cc:3037] T 098ba0adf30c47d49848f5428d7f3cc5 P 8b1a57b1c16a4f71a055f46154de9947 [term 0 FOLLOWER]: Advancing to term 1
I20260806 18:34:12.897076 21946 raft_consensus.cc:504] T 098ba0adf30c47d49848f5428d7f3cc5 P 8b1a57b1c16a4f71a055f46154de9947 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b1a57b1c16a4f71a055f46154de9947" member_type: VOTER last_known_addr { host: "127.19.194.3" port: 34037 } } peers { permanent_uuid: "ef292515d9c349a8a04b90e9476e31cd" member_type: VOTER last_known_addr { host: "127.19.194.1" port: 39693 } } peers { permanent_uuid: "4d22b7580880463a9383831743e285e8" member_type: VOTER last_known_addr { host: "127.19.194.2" port: 40521 } }
I20260806 18:34:12.897372 21946 leader_election.cc:288] T 098ba0adf30c47d49848f5428d7f3cc5 P 8b1a57b1c16a4f71a055f46154de9947 [CANDIDATE]: Term 1 election: Requested vote from peers ef292515d9c349a8a04b90e9476e31cd (127.19.194.1:39693), 4d22b7580880463a9383831743e285e8 (127.19.194.2:40521)
I20260806 18:34:12.897666 21578 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "098ba0adf30c47d49848f5428d7f3cc5" candidate_uuid: "8b1a57b1c16a4f71a055f46154de9947" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ef292515d9c349a8a04b90e9476e31cd"
I20260806 18:34:12.897687 21722 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "098ba0adf30c47d49848f5428d7f3cc5" candidate_uuid: "8b1a57b1c16a4f71a055f46154de9947" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4d22b7580880463a9383831743e285e8"
I20260806 18:34:12.897778 21722 raft_consensus.cc:3037] T 098ba0adf30c47d49848f5428d7f3cc5 P 4d22b7580880463a9383831743e285e8 [term 0 FOLLOWER]: Advancing to term 1
I20260806 18:34:12.897791 21578 raft_consensus.cc:3037] T 098ba0adf30c47d49848f5428d7f3cc5 P ef292515d9c349a8a04b90e9476e31cd [term 0 FOLLOWER]: Advancing to term 1
I20260806 18:34:12.899056 21578 raft_consensus.cc:2454] T 098ba0adf30c47d49848f5428d7f3cc5 P ef292515d9c349a8a04b90e9476e31cd [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8b1a57b1c16a4f71a055f46154de9947 in term 1.
I20260806 18:34:12.899243 21722 raft_consensus.cc:2454] T 098ba0adf30c47d49848f5428d7f3cc5 P 4d22b7580880463a9383831743e285e8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8b1a57b1c16a4f71a055f46154de9947 in term 1.
I20260806 18:34:12.899375 21800 leader_election.cc:302] T 098ba0adf30c47d49848f5428d7f3cc5 P 8b1a57b1c16a4f71a055f46154de9947 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8b1a57b1c16a4f71a055f46154de9947, ef292515d9c349a8a04b90e9476e31cd; no voters:
I20260806 18:34:12.899562 21946 raft_consensus.cc:2781] T 098ba0adf30c47d49848f5428d7f3cc5 P 8b1a57b1c16a4f71a055f46154de9947 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 18:34:12.899765 21946 raft_consensus.cc:686] T 098ba0adf30c47d49848f5428d7f3cc5 P 8b1a57b1c16a4f71a055f46154de9947 [term 1 LEADER]: Becoming Leader. State: Replica: 8b1a57b1c16a4f71a055f46154de9947, State: Running, Role: LEADER
I20260806 18:34:12.900058 21946 consensus_queue.cc:237] T 098ba0adf30c47d49848f5428d7f3cc5 P 8b1a57b1c16a4f71a055f46154de9947 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b1a57b1c16a4f71a055f46154de9947" member_type: VOTER last_known_addr { host: "127.19.194.3" port: 34037 } } peers { permanent_uuid: "ef292515d9c349a8a04b90e9476e31cd" member_type: VOTER last_known_addr { host: "127.19.194.1" port: 39693 } } peers { permanent_uuid: "4d22b7580880463a9383831743e285e8" member_type: VOTER last_known_addr { host: "127.19.194.2" port: 40521 } }
I20260806 18:34:12.902828 21405 catalog_manager.cc:5495] T 098ba0adf30c47d49848f5428d7f3cc5 P 8b1a57b1c16a4f71a055f46154de9947 reported cstate change: term changed from 0 to 1, leader changed from <none> to 8b1a57b1c16a4f71a055f46154de9947 (127.19.194.3). New cstate: current_term: 1 leader_uuid: "8b1a57b1c16a4f71a055f46154de9947" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b1a57b1c16a4f71a055f46154de9947" member_type: VOTER last_known_addr { host: "127.19.194.3" port: 34037 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ef292515d9c349a8a04b90e9476e31cd" member_type: VOTER last_known_addr { host: "127.19.194.1" port: 39693 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4d22b7580880463a9383831743e285e8" member_type: VOTER last_known_addr { host: "127.19.194.2" port: 40521 } health_report { overall_health: UNKNOWN } } }
I20260806 18:34:12.923578 21933 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:34:12.919711 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46374 (local address 127.19.194.3:34037)
0806 18:34:12.919835 (+ 124us) server_negotiation.cc:206] Beginning negotiation
0806 18:34:12.919839 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 18:34:12.919941 (+ 102us) server_negotiation.cc:407] Connection header received
0806 18:34:12.920176 (+ 235us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:34:12.920180 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:34:12.920236 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:34:12.920333 (+ 97us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 18:34:12.920589 (+ 256us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:34:12.921197 (+ 608us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:34:12.922071 (+ 874us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:34:12.922304 (+ 233us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:34:12.922407 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:34:12.922736 (+ 329us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 18:34:12.923087 (+ 351us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 18:34:12.923258 (+ 171us) server_negotiation.cc:1035] Waiting for connection context
0806 18:34:12.923367 (+ 109us) server_negotiation.cc:299] Negotiation successful
0806 18:34:12.923438 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260806 18:34:12.928871 21578 raft_consensus.cc:1261] T 098ba0adf30c47d49848f5428d7f3cc5 P ef292515d9c349a8a04b90e9476e31cd [term 1 FOLLOWER]: Refusing update from remote peer 8b1a57b1c16a4f71a055f46154de9947: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 18:34:12.928906 21722 raft_consensus.cc:1261] T 098ba0adf30c47d49848f5428d7f3cc5 P 4d22b7580880463a9383831743e285e8 [term 1 FOLLOWER]: Refusing update from remote peer 8b1a57b1c16a4f71a055f46154de9947: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 18:34:12.929517 21946 consensus_queue.cc:1035] T 098ba0adf30c47d49848f5428d7f3cc5 P 8b1a57b1c16a4f71a055f46154de9947 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ef292515d9c349a8a04b90e9476e31cd" member_type: VOTER last_known_addr { host: "127.19.194.1" port: 39693 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 18:34:12.929713 21949 consensus_queue.cc:1035] T 098ba0adf30c47d49848f5428d7f3cc5 P 8b1a57b1c16a4f71a055f46154de9947 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4d22b7580880463a9383831743e285e8" member_type: VOTER last_known_addr { host: "127.19.194.2" port: 40521 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 18:34:12.937062 21953 mvcc.cc:205] Tried to move back new op lower bound from 7315624971990441984 to 7315624971880919040. Current Snapshot: MvccSnapshot[applied={T|T < 7315624971990441984}]
I20260806 18:34:12.949194 21405 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:47880:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 18:34:12.949369 21405 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:47880:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 18:34:12.951776 21405 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 4bfe62b1286a4f54925c8a7b156a6adf: Sending DeleteTablet for 3 replicas of tablet 098ba0adf30c47d49848f5428d7f3cc5
I20260806 18:34:12.952672 21556 tablet_service.cc:1514] Processing DeleteTablet for tablet 098ba0adf30c47d49848f5428d7f3cc5 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 18:34:12 UTC) from {username='kudu', principal='kudu/127.19.194.62@KRBTEST.COM'} at 127.0.0.1:55038
I20260806 18:34:12.952654 21848 tablet_service.cc:1514] Processing DeleteTablet for tablet 098ba0adf30c47d49848f5428d7f3cc5 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 18:34:12 UTC) from {username='kudu', principal='kudu/127.19.194.62@KRBTEST.COM'} at 127.0.0.1:46366
I20260806 18:34:12.953156 21962 tablet_replica.cc:330] T 098ba0adf30c47d49848f5428d7f3cc5 P 8b1a57b1c16a4f71a055f46154de9947: stopping tablet replica
I20260806 18:34:12.953338 21702 tablet_service.cc:1514] Processing DeleteTablet for tablet 098ba0adf30c47d49848f5428d7f3cc5 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 18:34:12 UTC) from {username='kudu', principal='kudu/127.19.194.62@KRBTEST.COM'} at 127.0.0.1:47116
I20260806 18:34:12.953665 21962 raft_consensus.cc:2229] T 098ba0adf30c47d49848f5428d7f3cc5 P 8b1a57b1c16a4f71a055f46154de9947 [term 1 LEADER]: Raft consensus shutting down.
I20260806 18:34:12.953835 21963 tablet_replica.cc:330] T 098ba0adf30c47d49848f5428d7f3cc5 P ef292515d9c349a8a04b90e9476e31cd: stopping tablet replica
I20260806 18:34:12.953832 21405 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:47880:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260806 18:34:12.954123 21963 raft_consensus.cc:2229] T 098ba0adf30c47d49848f5428d7f3cc5 P ef292515d9c349a8a04b90e9476e31cd [term 1 FOLLOWER]: Raft consensus shutting down.
W20260806 18:34:12.954267 21405 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 18:34:12.954427 21963 raft_consensus.cc:2258] T 098ba0adf30c47d49848f5428d7f3cc5 P ef292515d9c349a8a04b90e9476e31cd [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 18:34:12.954447 21962 raft_consensus.cc:2258] T 098ba0adf30c47d49848f5428d7f3cc5 P 8b1a57b1c16a4f71a055f46154de9947 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 18:34:12.955981 21963 ts_tablet_manager.cc:1905] T 098ba0adf30c47d49848f5428d7f3cc5 P ef292515d9c349a8a04b90e9476e31cd: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 18:34:12.956154 21962 ts_tablet_manager.cc:1905] T 098ba0adf30c47d49848f5428d7f3cc5 P 8b1a57b1c16a4f71a055f46154de9947: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 18:34:12.958786 21963 ts_tablet_manager.cc:1918] T 098ba0adf30c47d49848f5428d7f3cc5 P ef292515d9c349a8a04b90e9476e31cd: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 18:34:12.958889 21963 log.cc:1196] T 098ba0adf30c47d49848f5428d7f3cc5 P ef292515d9c349a8a04b90e9476e31cd: Deleting WAL directory at /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786041246094603-20232-0/minicluster-data/ts-0/wal/wals/098ba0adf30c47d49848f5428d7f3cc5
I20260806 18:34:12.959142 21962 ts_tablet_manager.cc:1918] T 098ba0adf30c47d49848f5428d7f3cc5 P 8b1a57b1c16a4f71a055f46154de9947: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 18:34:12.959344 21962 log.cc:1196] T 098ba0adf30c47d49848f5428d7f3cc5 P 8b1a57b1c16a4f71a055f46154de9947: Deleting WAL directory at /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786041246094603-20232-0/minicluster-data/ts-2/wal/wals/098ba0adf30c47d49848f5428d7f3cc5
I20260806 18:34:12.959744 21962 ts_tablet_manager.cc:1939] T 098ba0adf30c47d49848f5428d7f3cc5 P 8b1a57b1c16a4f71a055f46154de9947: Deleting consensus metadata
I20260806 18:34:12.959692 21702 tablet_service.cc:1467] Processing CreateTablet for tablet cfa8797235e94ae9be65666f05e8f629 (DEFAULT_TABLE table=test-table [id=cbff0f0c226f4de3b11d42281d252c60]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 18:34:12.959730 21556 tablet_service.cc:1467] Processing CreateTablet for tablet cfa8797235e94ae9be65666f05e8f629 (DEFAULT_TABLE table=test-table [id=cbff0f0c226f4de3b11d42281d252c60]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 18:34:12.959959 21702 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet cfa8797235e94ae9be65666f05e8f629. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 18:34:12.960005 21556 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet cfa8797235e94ae9be65666f05e8f629. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 18:34:12.961015 21391 catalog_manager.cc:4841] TS 8b1a57b1c16a4f71a055f46154de9947 (127.19.194.3:34037): tablet 098ba0adf30c47d49848f5428d7f3cc5 (table test-table [id=d6bdfac9aa3849beb05bfb11bc2f1fd5]) successfully deleted
I20260806 18:34:12.962077 21938 tablet_bootstrap.cc:492] T cfa8797235e94ae9be65666f05e8f629 P ef292515d9c349a8a04b90e9476e31cd: Bootstrap starting.
I20260806 18:34:12.962251 21940 tablet_bootstrap.cc:492] T cfa8797235e94ae9be65666f05e8f629 P 4d22b7580880463a9383831743e285e8: Bootstrap starting.
I20260806 18:34:12.962466 21848 tablet_service.cc:1467] Processing CreateTablet for tablet cfa8797235e94ae9be65666f05e8f629 (DEFAULT_TABLE table=test-table [id=cbff0f0c226f4de3b11d42281d252c60]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 18:34:12.962705 21848 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet cfa8797235e94ae9be65666f05e8f629. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 18:34:12.963276 21940 tablet_bootstrap.cc:654] T cfa8797235e94ae9be65666f05e8f629 P 4d22b7580880463a9383831743e285e8: Neither blocks nor log segments found. Creating new log.
I20260806 18:34:12.963544 21964 tablet_replica.cc:330] T 098ba0adf30c47d49848f5428d7f3cc5 P 4d22b7580880463a9383831743e285e8: stopping tablet replica
I20260806 18:34:12.963836 21964 raft_consensus.cc:2229] T 098ba0adf30c47d49848f5428d7f3cc5 P 4d22b7580880463a9383831743e285e8 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 18:34:12.964129 21963 ts_tablet_manager.cc:1939] T 098ba0adf30c47d49848f5428d7f3cc5 P ef292515d9c349a8a04b90e9476e31cd: Deleting consensus metadata
I20260806 18:34:12.964177 21964 raft_consensus.cc:2258] T 098ba0adf30c47d49848f5428d7f3cc5 P 4d22b7580880463a9383831743e285e8 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 18:34:12.964228 21938 tablet_bootstrap.cc:654] T cfa8797235e94ae9be65666f05e8f629 P ef292515d9c349a8a04b90e9476e31cd: Neither blocks nor log segments found. Creating new log.
I20260806 18:34:12.965480 21390 catalog_manager.cc:4841] TS ef292515d9c349a8a04b90e9476e31cd (127.19.194.1:39693): tablet 098ba0adf30c47d49848f5428d7f3cc5 (table test-table [id=d6bdfac9aa3849beb05bfb11bc2f1fd5]) successfully deleted
I20260806 18:34:12.965571 21939 tablet_bootstrap.cc:492] T cfa8797235e94ae9be65666f05e8f629 P 8b1a57b1c16a4f71a055f46154de9947: Bootstrap starting.
I20260806 18:34:12.966488 21939 tablet_bootstrap.cc:654] T cfa8797235e94ae9be65666f05e8f629 P 8b1a57b1c16a4f71a055f46154de9947: Neither blocks nor log segments found. Creating new log.
I20260806 18:34:12.966502 21964 ts_tablet_manager.cc:1905] T 098ba0adf30c47d49848f5428d7f3cc5 P 4d22b7580880463a9383831743e285e8: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 18:34:12.967856 21938 tablet_bootstrap.cc:492] T cfa8797235e94ae9be65666f05e8f629 P ef292515d9c349a8a04b90e9476e31cd: No bootstrap required, opened a new log
I20260806 18:34:12.967935 21938 ts_tablet_manager.cc:1397] T cfa8797235e94ae9be65666f05e8f629 P ef292515d9c349a8a04b90e9476e31cd: Time spent bootstrapping tablet: real 0.006s user 0.003s sys 0.000s
I20260806 18:34:12.968349 21938 raft_consensus.cc:348] T cfa8797235e94ae9be65666f05e8f629 P ef292515d9c349a8a04b90e9476e31cd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef292515d9c349a8a04b90e9476e31cd" member_type: VOTER last_known_addr { host: "127.19.194.1" port: 39693 } } peers { permanent_uuid: "4d22b7580880463a9383831743e285e8" member_type: VOTER last_known_addr { host: "127.19.194.2" port: 40521 } } peers { permanent_uuid: "8b1a57b1c16a4f71a055f46154de9947" member_type: VOTER last_known_addr { host: "127.19.194.3" port: 34037 } }
I20260806 18:34:12.968451 21938 raft_consensus.cc:374] T cfa8797235e94ae9be65666f05e8f629 P ef292515d9c349a8a04b90e9476e31cd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 18:34:12.968492 21938 raft_consensus.cc:729] T cfa8797235e94ae9be65666f05e8f629 P ef292515d9c349a8a04b90e9476e31cd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ef292515d9c349a8a04b90e9476e31cd, State: Initialized, Role: FOLLOWER
I20260806 18:34:12.968622 21938 consensus_queue.cc:260] T cfa8797235e94ae9be65666f05e8f629 P ef292515d9c349a8a04b90e9476e31cd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef292515d9c349a8a04b90e9476e31cd" member_type: VOTER last_known_addr { host: "127.19.194.1" port: 39693 } } peers { permanent_uuid: "4d22b7580880463a9383831743e285e8" member_type: VOTER last_known_addr { host: "127.19.194.2" port: 40521 } } peers { permanent_uuid: "8b1a57b1c16a4f71a055f46154de9947" member_type: VOTER last_known_addr { host: "127.19.194.3" port: 34037 } }
I20260806 18:34:12.968856 21938 ts_tablet_manager.cc:1428] T cfa8797235e94ae9be65666f05e8f629 P ef292515d9c349a8a04b90e9476e31cd: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20260806 18:34:12.969501 21939 tablet_bootstrap.cc:492] T cfa8797235e94ae9be65666f05e8f629 P 8b1a57b1c16a4f71a055f46154de9947: No bootstrap required, opened a new log
I20260806 18:34:12.969589 21939 ts_tablet_manager.cc:1397] T cfa8797235e94ae9be65666f05e8f629 P 8b1a57b1c16a4f71a055f46154de9947: Time spent bootstrapping tablet: real 0.004s user 0.001s sys 0.001s
I20260806 18:34:12.970059 21939 raft_consensus.cc:348] T cfa8797235e94ae9be65666f05e8f629 P 8b1a57b1c16a4f71a055f46154de9947 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef292515d9c349a8a04b90e9476e31cd" member_type: VOTER last_known_addr { host: "127.19.194.1" port: 39693 } } peers { permanent_uuid: "4d22b7580880463a9383831743e285e8" member_type: VOTER last_known_addr { host: "127.19.194.2" port: 40521 } } peers { permanent_uuid: "8b1a57b1c16a4f71a055f46154de9947" member_type: VOTER last_known_addr { host: "127.19.194.3" port: 34037 } }
I20260806 18:34:12.970163 21939 raft_consensus.cc:374] T cfa8797235e94ae9be65666f05e8f629 P 8b1a57b1c16a4f71a055f46154de9947 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 18:34:12.970192 21939 raft_consensus.cc:729] T cfa8797235e94ae9be65666f05e8f629 P 8b1a57b1c16a4f71a055f46154de9947 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8b1a57b1c16a4f71a055f46154de9947, State: Initialized, Role: FOLLOWER
I20260806 18:34:12.970293 21939 consensus_queue.cc:260] T cfa8797235e94ae9be65666f05e8f629 P 8b1a57b1c16a4f71a055f46154de9947 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef292515d9c349a8a04b90e9476e31cd" member_type: VOTER last_known_addr { host: "127.19.194.1" port: 39693 } } peers { permanent_uuid: "4d22b7580880463a9383831743e285e8" member_type: VOTER last_known_addr { host: "127.19.194.2" port: 40521 } } peers { permanent_uuid: "8b1a57b1c16a4f71a055f46154de9947" member_type: VOTER last_known_addr { host: "127.19.194.3" port: 34037 } }
I20260806 18:34:12.970527 21939 ts_tablet_manager.cc:1428] T cfa8797235e94ae9be65666f05e8f629 P 8b1a57b1c16a4f71a055f46154de9947: Time spent starting tablet: real 0.001s user 0.001s sys 0.001s
I20260806 18:34:12.971736 21964 ts_tablet_manager.cc:1918] T 098ba0adf30c47d49848f5428d7f3cc5 P 4d22b7580880463a9383831743e285e8: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 18:34:12.971958 21964 log.cc:1196] T 098ba0adf30c47d49848f5428d7f3cc5 P 4d22b7580880463a9383831743e285e8: Deleting WAL directory at /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786041246094603-20232-0/minicluster-data/ts-1/wal/wals/098ba0adf30c47d49848f5428d7f3cc5
I20260806 18:34:12.972316 21964 ts_tablet_manager.cc:1939] T 098ba0adf30c47d49848f5428d7f3cc5 P 4d22b7580880463a9383831743e285e8: Deleting consensus metadata
I20260806 18:34:12.973466 21391 catalog_manager.cc:4841] TS 4d22b7580880463a9383831743e285e8 (127.19.194.2:40521): tablet 098ba0adf30c47d49848f5428d7f3cc5 (table test-table [id=d6bdfac9aa3849beb05bfb11bc2f1fd5]) successfully deleted
I20260806 18:34:12.974109 21940 tablet_bootstrap.cc:492] T cfa8797235e94ae9be65666f05e8f629 P 4d22b7580880463a9383831743e285e8: No bootstrap required, opened a new log
I20260806 18:34:12.974201 21940 ts_tablet_manager.cc:1397] T cfa8797235e94ae9be65666f05e8f629 P 4d22b7580880463a9383831743e285e8: Time spent bootstrapping tablet: real 0.012s user 0.000s sys 0.002s
I20260806 18:34:12.974632 21940 raft_consensus.cc:348] T cfa8797235e94ae9be65666f05e8f629 P 4d22b7580880463a9383831743e285e8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef292515d9c349a8a04b90e9476e31cd" member_type: VOTER last_known_addr { host: "127.19.194.1" port: 39693 } } peers { permanent_uuid: "4d22b7580880463a9383831743e285e8" member_type: VOTER last_known_addr { host: "127.19.194.2" port: 40521 } } peers { permanent_uuid: "8b1a57b1c16a4f71a055f46154de9947" member_type: VOTER last_known_addr { host: "127.19.194.3" port: 34037 } }
I20260806 18:34:12.974742 21940 raft_consensus.cc:374] T cfa8797235e94ae9be65666f05e8f629 P 4d22b7580880463a9383831743e285e8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 18:34:12.974774 21940 raft_consensus.cc:729] T cfa8797235e94ae9be65666f05e8f629 P 4d22b7580880463a9383831743e285e8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4d22b7580880463a9383831743e285e8, State: Initialized, Role: FOLLOWER
I20260806 18:34:12.974891 21940 consensus_queue.cc:260] T cfa8797235e94ae9be65666f05e8f629 P 4d22b7580880463a9383831743e285e8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef292515d9c349a8a04b90e9476e31cd" member_type: VOTER last_known_addr { host: "127.19.194.1" port: 39693 } } peers { permanent_uuid: "4d22b7580880463a9383831743e285e8" member_type: VOTER last_known_addr { host: "127.19.194.2" port: 40521 } } peers { permanent_uuid: "8b1a57b1c16a4f71a055f46154de9947" member_type: VOTER last_known_addr { host: "127.19.194.3" port: 34037 } }
I20260806 18:34:12.975176 21940 ts_tablet_manager.cc:1428] T cfa8797235e94ae9be65666f05e8f629 P 4d22b7580880463a9383831743e285e8: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20260806 18:34:13.029975 21957 raft_consensus.cc:482] T cfa8797235e94ae9be65666f05e8f629 P 8b1a57b1c16a4f71a055f46154de9947 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 18:34:13.030150 21957 raft_consensus.cc:504] T cfa8797235e94ae9be65666f05e8f629 P 8b1a57b1c16a4f71a055f46154de9947 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef292515d9c349a8a04b90e9476e31cd" member_type: VOTER last_known_addr { host: "127.19.194.1" port: 39693 } } peers { permanent_uuid: "4d22b7580880463a9383831743e285e8" member_type: VOTER last_known_addr { host: "127.19.194.2" port: 40521 } } peers { permanent_uuid: "8b1a57b1c16a4f71a055f46154de9947" member_type: VOTER last_known_addr { host: "127.19.194.3" port: 34037 } }
I20260806 18:34:13.030635 21957 leader_election.cc:288] T cfa8797235e94ae9be65666f05e8f629 P 8b1a57b1c16a4f71a055f46154de9947 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ef292515d9c349a8a04b90e9476e31cd (127.19.194.1:39693), 4d22b7580880463a9383831743e285e8 (127.19.194.2:40521)
I20260806 18:34:13.031023 21578 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cfa8797235e94ae9be65666f05e8f629" candidate_uuid: "8b1a57b1c16a4f71a055f46154de9947" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ef292515d9c349a8a04b90e9476e31cd" is_pre_election: true
I20260806 18:34:13.031088 21722 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cfa8797235e94ae9be65666f05e8f629" candidate_uuid: "8b1a57b1c16a4f71a055f46154de9947" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4d22b7580880463a9383831743e285e8" is_pre_election: true
I20260806 18:34:13.031206 21578 raft_consensus.cc:2454] T cfa8797235e94ae9be65666f05e8f629 P ef292515d9c349a8a04b90e9476e31cd [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8b1a57b1c16a4f71a055f46154de9947 in term 0.
I20260806 18:34:13.031289 21722 raft_consensus.cc:2454] T cfa8797235e94ae9be65666f05e8f629 P 4d22b7580880463a9383831743e285e8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8b1a57b1c16a4f71a055f46154de9947 in term 0.
I20260806 18:34:13.031584 21800 leader_election.cc:302] T cfa8797235e94ae9be65666f05e8f629 P 8b1a57b1c16a4f71a055f46154de9947 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 4d22b7580880463a9383831743e285e8, 8b1a57b1c16a4f71a055f46154de9947, ef292515d9c349a8a04b90e9476e31cd; no voters:
I20260806 18:34:13.031749 21957 raft_consensus.cc:2781] T cfa8797235e94ae9be65666f05e8f629 P 8b1a57b1c16a4f71a055f46154de9947 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 18:34:13.031811 21957 raft_consensus.cc:482] T cfa8797235e94ae9be65666f05e8f629 P 8b1a57b1c16a4f71a055f46154de9947 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 18:34:13.031853 21957 raft_consensus.cc:3037] T cfa8797235e94ae9be65666f05e8f629 P 8b1a57b1c16a4f71a055f46154de9947 [term 0 FOLLOWER]: Advancing to term 1
I20260806 18:34:13.032964 21957 raft_consensus.cc:504] T cfa8797235e94ae9be65666f05e8f629 P 8b1a57b1c16a4f71a055f46154de9947 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef292515d9c349a8a04b90e9476e31cd" member_type: VOTER last_known_addr { host: "127.19.194.1" port: 39693 } } peers { permanent_uuid: "4d22b7580880463a9383831743e285e8" member_type: VOTER last_known_addr { host: "127.19.194.2" port: 40521 } } peers { permanent_uuid: "8b1a57b1c16a4f71a055f46154de9947" member_type: VOTER last_known_addr { host: "127.19.194.3" port: 34037 } }
I20260806 18:34:13.033337 21957 leader_election.cc:288] T cfa8797235e94ae9be65666f05e8f629 P 8b1a57b1c16a4f71a055f46154de9947 [CANDIDATE]: Term 1 election: Requested vote from peers ef292515d9c349a8a04b90e9476e31cd (127.19.194.1:39693), 4d22b7580880463a9383831743e285e8 (127.19.194.2:40521)
I20260806 18:34:13.033653 21722 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cfa8797235e94ae9be65666f05e8f629" candidate_uuid: "8b1a57b1c16a4f71a055f46154de9947" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4d22b7580880463a9383831743e285e8"
I20260806 18:34:13.033663 21578 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cfa8797235e94ae9be65666f05e8f629" candidate_uuid: "8b1a57b1c16a4f71a055f46154de9947" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ef292515d9c349a8a04b90e9476e31cd"
I20260806 18:34:13.033766 21578 raft_consensus.cc:3037] T cfa8797235e94ae9be65666f05e8f629 P ef292515d9c349a8a04b90e9476e31cd [term 0 FOLLOWER]: Advancing to term 1
I20260806 18:34:13.033780 21722 raft_consensus.cc:3037] T cfa8797235e94ae9be65666f05e8f629 P 4d22b7580880463a9383831743e285e8 [term 0 FOLLOWER]: Advancing to term 1
I20260806 18:34:13.034677 21722 raft_consensus.cc:2454] T cfa8797235e94ae9be65666f05e8f629 P 4d22b7580880463a9383831743e285e8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8b1a57b1c16a4f71a055f46154de9947 in term 1.
I20260806 18:34:13.034951 21801 leader_election.cc:302] T cfa8797235e94ae9be65666f05e8f629 P 8b1a57b1c16a4f71a055f46154de9947 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4d22b7580880463a9383831743e285e8, 8b1a57b1c16a4f71a055f46154de9947; no voters:
I20260806 18:34:13.034974 21578 raft_consensus.cc:2454] T cfa8797235e94ae9be65666f05e8f629 P ef292515d9c349a8a04b90e9476e31cd [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8b1a57b1c16a4f71a055f46154de9947 in term 1.
I20260806 18:34:13.035175 21957 raft_consensus.cc:2781] T cfa8797235e94ae9be65666f05e8f629 P 8b1a57b1c16a4f71a055f46154de9947 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 18:34:13.035267 21957 raft_consensus.cc:686] T cfa8797235e94ae9be65666f05e8f629 P 8b1a57b1c16a4f71a055f46154de9947 [term 1 LEADER]: Becoming Leader. State: Replica: 8b1a57b1c16a4f71a055f46154de9947, State: Running, Role: LEADER
I20260806 18:34:13.035442 21957 consensus_queue.cc:237] T cfa8797235e94ae9be65666f05e8f629 P 8b1a57b1c16a4f71a055f46154de9947 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef292515d9c349a8a04b90e9476e31cd" member_type: VOTER last_known_addr { host: "127.19.194.1" port: 39693 } } peers { permanent_uuid: "4d22b7580880463a9383831743e285e8" member_type: VOTER last_known_addr { host: "127.19.194.2" port: 40521 } } peers { permanent_uuid: "8b1a57b1c16a4f71a055f46154de9947" member_type: VOTER last_known_addr { host: "127.19.194.3" port: 34037 } }
I20260806 18:34:13.037271 21406 catalog_manager.cc:5495] T cfa8797235e94ae9be65666f05e8f629 P 8b1a57b1c16a4f71a055f46154de9947 reported cstate change: term changed from 0 to 1, leader changed from <none> to 8b1a57b1c16a4f71a055f46154de9947 (127.19.194.3). New cstate: current_term: 1 leader_uuid: "8b1a57b1c16a4f71a055f46154de9947" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef292515d9c349a8a04b90e9476e31cd" member_type: VOTER last_known_addr { host: "127.19.194.1" port: 39693 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4d22b7580880463a9383831743e285e8" member_type: VOTER last_known_addr { host: "127.19.194.2" port: 40521 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8b1a57b1c16a4f71a055f46154de9947" member_type: VOTER last_known_addr { host: "127.19.194.3" port: 34037 } health_report { overall_health: HEALTHY } } }
W20260806 18:34:13.054318 21915 tablet.cc:2369] T cfa8797235e94ae9be65666f05e8f629 P 8b1a57b1c16a4f71a055f46154de9947: Can't schedule compaction. Clean time has not been advanced past its initial value.
Aug 06 18:34:13 dist-test-slave-1xrh krb5kdc[21356](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041252, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.194.62@KRBTEST.COM for kudu/127.19.194.62@KRBTEST.COM
I20260806 18:34:13.077926 21974 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:34:13.068788 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.194.62:34683 (local address 127.0.0.1:47894)
0806 18:34:13.069073 (+ 285us) negotiation.cc:107] Waiting for socket to connect
0806 18:34:13.069089 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0806 18:34:13.069222 (+ 133us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:34:13.069499 (+ 277us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:34:13.069502 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:34:13.069864 (+ 362us) client_negotiation.cc:189] Negotiated authn=SASL
0806 18:34:13.069959 (+ 95us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:34:13.069964 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:34:13.071001 (+ 1037us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:34:13.071003 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:34:13.071599 (+ 596us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:34:13.071607 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:34:13.072148 (+ 541us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:34:13.072151 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:34:13.072258 (+ 107us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:34:13.072837 (+ 579us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 18:34:13.072849 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0806 18:34:13.074542 (+ 1693us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 18:34:13.076444 (+ 1902us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:34:13.076452 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:34:13.076456 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:34:13.076667 (+ 211us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:34:13.076947 (+ 280us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:34:13.076948 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:34:13.076950 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:34:13.077043 (+ 93us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:34:13.077497 (+ 454us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 18:34:13.077502 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 18:34:13.077603 (+ 101us) client_negotiation.cc:769] Sending connection context
0806 18:34:13.077689 (+ 86us) client_negotiation.cc:240] Negotiation successful
0806 18:34:13.077793 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":205,"mutex_wait_us":32,"thread_start_us":109,"threads_started":1}
I20260806 18:34:13.078086 21460 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:34:13.068889 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47894 (local address 127.19.194.62:34683)
0806 18:34:13.069022 (+ 133us) server_negotiation.cc:206] Beginning negotiation
0806 18:34:13.069027 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 18:34:13.069171 (+ 144us) server_negotiation.cc:407] Connection header received
0806 18:34:13.069312 (+ 141us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:34:13.069316 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:34:13.069369 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:34:13.069468 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:34:13.070062 (+ 594us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:34:13.070894 (+ 832us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:34:13.071728 (+ 834us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:34:13.072033 (+ 305us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:34:13.072151 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:34:13.074687 (+ 2536us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:34:13.074691 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:34:13.074694 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 18:34:13.074719 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:34:13.076342 (+ 1623us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:34:13.076767 (+ 425us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:34:13.076770 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:34:13.076772 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:34:13.076859 (+ 87us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:34:13.077136 (+ 277us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:34:13.077142 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:34:13.077144 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:34:13.077389 (+ 245us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:34:13.077507 (+ 118us) server_negotiation.cc:1035] Waiting for connection context
0806 18:34:13.077775 (+ 268us) server_negotiation.cc:299] Negotiation successful
0806 18:34:13.077903 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":29,"server-negotiator.queue_time_us":44}
I20260806 18:34:13.081820 21406 catalog_manager.cc:1908] Servicing CreateTable request from {username='kudu', principal='kudu/127.19.194.62@KRBTEST.COM'} at 127.0.0.1:47894:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260806 18:34:13.082615 21406 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20260806 18:34:13.083897 21773 tablet.cc:2369] T cfa8797235e94ae9be65666f05e8f629 P 4d22b7580880463a9383831743e285e8: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 18:34:13.087641 21556 tablet_service.cc:1467] Processing CreateTablet for tablet 2d63df7dd787429089261e8739a0a9fc (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=1b85bc5abc6f474eb4560aa7b90408f2]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260806 18:34:13.087815 21848 tablet_service.cc:1467] Processing CreateTablet for tablet 2d63df7dd787429089261e8739a0a9fc (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=1b85bc5abc6f474eb4560aa7b90408f2]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260806 18:34:13.088019 21556 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2d63df7dd787429089261e8739a0a9fc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 18:34:13.088218 21848 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2d63df7dd787429089261e8739a0a9fc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 18:34:13.090448 21939 tablet_bootstrap.cc:492] T 2d63df7dd787429089261e8739a0a9fc P 8b1a57b1c16a4f71a055f46154de9947: Bootstrap starting.
I20260806 18:34:13.090759 21702 tablet_service.cc:1467] Processing CreateTablet for tablet 2d63df7dd787429089261e8739a0a9fc (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=1b85bc5abc6f474eb4560aa7b90408f2]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260806 18:34:13.091054 21938 tablet_bootstrap.cc:492] T 2d63df7dd787429089261e8739a0a9fc P ef292515d9c349a8a04b90e9476e31cd: Bootstrap starting.
I20260806 18:34:13.091166 21702 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2d63df7dd787429089261e8739a0a9fc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 18:34:13.091565 21939 tablet_bootstrap.cc:654] T 2d63df7dd787429089261e8739a0a9fc P 8b1a57b1c16a4f71a055f46154de9947: Neither blocks nor log segments found. Creating new log.
I20260806 18:34:13.092008 21938 tablet_bootstrap.cc:654] T 2d63df7dd787429089261e8739a0a9fc P ef292515d9c349a8a04b90e9476e31cd: Neither blocks nor log segments found. Creating new log.
I20260806 18:34:13.093271 21938 tablet_bootstrap.cc:492] T 2d63df7dd787429089261e8739a0a9fc P ef292515d9c349a8a04b90e9476e31cd: No bootstrap required, opened a new log
I20260806 18:34:13.093359 21938 ts_tablet_manager.cc:1397] T 2d63df7dd787429089261e8739a0a9fc P ef292515d9c349a8a04b90e9476e31cd: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260806 18:34:13.093791 21938 raft_consensus.cc:348] T 2d63df7dd787429089261e8739a0a9fc P ef292515d9c349a8a04b90e9476e31cd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef292515d9c349a8a04b90e9476e31cd" member_type: VOTER last_known_addr { host: "127.19.194.1" port: 39693 } } peers { permanent_uuid: "4d22b7580880463a9383831743e285e8" member_type: VOTER last_known_addr { host: "127.19.194.2" port: 40521 } } peers { permanent_uuid: "8b1a57b1c16a4f71a055f46154de9947" member_type: VOTER last_known_addr { host: "127.19.194.3" port: 34037 } }
I20260806 18:34:13.093943 21938 raft_consensus.cc:374] T 2d63df7dd787429089261e8739a0a9fc P ef292515d9c349a8a04b90e9476e31cd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 18:34:13.093976 21938 raft_consensus.cc:729] T 2d63df7dd787429089261e8739a0a9fc P ef292515d9c349a8a04b90e9476e31cd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ef292515d9c349a8a04b90e9476e31cd, State: Initialized, Role: FOLLOWER
I20260806 18:34:13.094161 21940 tablet_bootstrap.cc:492] T 2d63df7dd787429089261e8739a0a9fc P 4d22b7580880463a9383831743e285e8: Bootstrap starting.
I20260806 18:34:13.095103 21940 tablet_bootstrap.cc:654] T 2d63df7dd787429089261e8739a0a9fc P 4d22b7580880463a9383831743e285e8: Neither blocks nor log segments found. Creating new log.
I20260806 18:34:13.095500 21939 tablet_bootstrap.cc:492] T 2d63df7dd787429089261e8739a0a9fc P 8b1a57b1c16a4f71a055f46154de9947: No bootstrap required, opened a new log
I20260806 18:34:13.095610 21939 ts_tablet_manager.cc:1397] T 2d63df7dd787429089261e8739a0a9fc P 8b1a57b1c16a4f71a055f46154de9947: Time spent bootstrapping tablet: real 0.005s user 0.002s sys 0.000s
I20260806 18:34:13.095748 21938 consensus_queue.cc:260] T 2d63df7dd787429089261e8739a0a9fc P ef292515d9c349a8a04b90e9476e31cd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef292515d9c349a8a04b90e9476e31cd" member_type: VOTER last_known_addr { host: "127.19.194.1" port: 39693 } } peers { permanent_uuid: "4d22b7580880463a9383831743e285e8" member_type: VOTER last_known_addr { host: "127.19.194.2" port: 40521 } } peers { permanent_uuid: "8b1a57b1c16a4f71a055f46154de9947" member_type: VOTER last_known_addr { host: "127.19.194.3" port: 34037 } }
I20260806 18:34:13.096045 21938 ts_tablet_manager.cc:1428] T 2d63df7dd787429089261e8739a0a9fc P ef292515d9c349a8a04b90e9476e31cd: Time spent starting tablet: real 0.003s user 0.001s sys 0.000s
I20260806 18:34:13.096081 21939 raft_consensus.cc:348] T 2d63df7dd787429089261e8739a0a9fc P 8b1a57b1c16a4f71a055f46154de9947 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef292515d9c349a8a04b90e9476e31cd" member_type: VOTER last_known_addr { host: "127.19.194.1" port: 39693 } } peers { permanent_uuid: "4d22b7580880463a9383831743e285e8" member_type: VOTER last_known_addr { host: "127.19.194.2" port: 40521 } } peers { permanent_uuid: "8b1a57b1c16a4f71a055f46154de9947" member_type: VOTER last_known_addr { host: "127.19.194.3" port: 34037 } }
I20260806 18:34:13.096187 21939 raft_consensus.cc:374] T 2d63df7dd787429089261e8739a0a9fc P 8b1a57b1c16a4f71a055f46154de9947 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 18:34:13.096220 21939 raft_consensus.cc:729] T 2d63df7dd787429089261e8739a0a9fc P 8b1a57b1c16a4f71a055f46154de9947 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8b1a57b1c16a4f71a055f46154de9947, State: Initialized, Role: FOLLOWER
I20260806 18:34:13.096333 21939 consensus_queue.cc:260] T 2d63df7dd787429089261e8739a0a9fc P 8b1a57b1c16a4f71a055f46154de9947 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef292515d9c349a8a04b90e9476e31cd" member_type: VOTER last_known_addr { host: "127.19.194.1" port: 39693 } } peers { permanent_uuid: "4d22b7580880463a9383831743e285e8" member_type: VOTER last_known_addr { host: "127.19.194.2" port: 40521 } } peers { permanent_uuid: "8b1a57b1c16a4f71a055f46154de9947" member_type: VOTER last_known_addr { host: "127.19.194.3" port: 34037 } }
I20260806 18:34:13.096611 21939 ts_tablet_manager.cc:1428] T 2d63df7dd787429089261e8739a0a9fc P 8b1a57b1c16a4f71a055f46154de9947: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20260806 18:34:13.096561 21946 tablet_replica.cc:439] T 2d63df7dd787429089261e8739a0a9fc P 8b1a57b1c16a4f71a055f46154de9947: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef292515d9c349a8a04b90e9476e31cd" member_type: VOTER last_known_addr { host: "127.19.194.1" port: 39693 } } peers { permanent_uuid: "4d22b7580880463a9383831743e285e8" member_type: VOTER last_known_addr { host: "127.19.194.2" port: 40521 } } peers { permanent_uuid: "8b1a57b1c16a4f71a055f46154de9947" member_type: VOTER last_known_addr { host: "127.19.194.3" port: 34037 } } }
I20260806 18:34:13.096683 21946 tablet_replica.cc:442] T 2d63df7dd787429089261e8739a0a9fc P 8b1a57b1c16a4f71a055f46154de9947: This TxnStatusTablet replica's current role is: FOLLOWER
I20260806 18:34:13.096779 21940 tablet_bootstrap.cc:492] T 2d63df7dd787429089261e8739a0a9fc P 4d22b7580880463a9383831743e285e8: No bootstrap required, opened a new log
I20260806 18:34:13.096849 21940 ts_tablet_manager.cc:1397] T 2d63df7dd787429089261e8739a0a9fc P 4d22b7580880463a9383831743e285e8: Time spent bootstrapping tablet: real 0.003s user 0.000s sys 0.001s
I20260806 18:34:13.097276 21940 raft_consensus.cc:348] T 2d63df7dd787429089261e8739a0a9fc P 4d22b7580880463a9383831743e285e8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef292515d9c349a8a04b90e9476e31cd" member_type: VOTER last_known_addr { host: "127.19.194.1" port: 39693 } } peers { permanent_uuid: "4d22b7580880463a9383831743e285e8" member_type: VOTER last_known_addr { host: "127.19.194.2" port: 40521 } } peers { permanent_uuid: "8b1a57b1c16a4f71a055f46154de9947" member_type: VOTER last_known_addr { host: "127.19.194.3" port: 34037 } }
I20260806 18:34:13.096048 21944 tablet_replica.cc:439] T 2d63df7dd787429089261e8739a0a9fc P ef292515d9c349a8a04b90e9476e31cd: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef292515d9c349a8a04b90e9476e31cd" member_type: VOTER last_known_addr { host: "127.19.194.1" port: 39693 } } peers { permanent_uuid: "4d22b7580880463a9383831743e285e8" member_type: VOTER last_known_addr { host: "127.19.194.2" port: 40521 } } peers { permanent_uuid: "8b1a57b1c16a4f71a055f46154de9947" member_type: VOTER last_known_addr { host: "127.19.194.3" port: 34037 } } }
I20260806 18:34:13.097393 21940 raft_consensus.cc:374] T 2d63df7dd787429089261e8739a0a9fc P 4d22b7580880463a9383831743e285e8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 18:34:13.097416 21944 tablet_replica.cc:442] T 2d63df7dd787429089261e8739a0a9fc P ef292515d9c349a8a04b90e9476e31cd: This TxnStatusTablet replica's current role is: FOLLOWER
I20260806 18:34:13.097424 21940 raft_consensus.cc:729] T 2d63df7dd787429089261e8739a0a9fc P 4d22b7580880463a9383831743e285e8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4d22b7580880463a9383831743e285e8, State: Initialized, Role: FOLLOWER
I20260806 18:34:13.097612 21940 consensus_queue.cc:260] T 2d63df7dd787429089261e8739a0a9fc P 4d22b7580880463a9383831743e285e8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef292515d9c349a8a04b90e9476e31cd" member_type: VOTER last_known_addr { host: "127.19.194.1" port: 39693 } } peers { permanent_uuid: "4d22b7580880463a9383831743e285e8" member_type: VOTER last_known_addr { host: "127.19.194.2" port: 40521 } } peers { permanent_uuid: "8b1a57b1c16a4f71a055f46154de9947" member_type: VOTER last_known_addr { host: "127.19.194.3" port: 34037 } }
I20260806 18:34:13.097970 21940 ts_tablet_manager.cc:1428] T 2d63df7dd787429089261e8739a0a9fc P 4d22b7580880463a9383831743e285e8: Time spent starting tablet: real 0.001s user 0.001s sys 0.001s
I20260806 18:34:13.098205 21945 tablet_replica.cc:439] T 2d63df7dd787429089261e8739a0a9fc P 4d22b7580880463a9383831743e285e8: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef292515d9c349a8a04b90e9476e31cd" member_type: VOTER last_known_addr { host: "127.19.194.1" port: 39693 } } peers { permanent_uuid: "4d22b7580880463a9383831743e285e8" member_type: VOTER last_known_addr { host: "127.19.194.2" port: 40521 } } peers { permanent_uuid: "8b1a57b1c16a4f71a055f46154de9947" member_type: VOTER last_known_addr { host: "127.19.194.3" port: 34037 } } }
I20260806 18:34:13.098318 21945 tablet_replica.cc:442] T 2d63df7dd787429089261e8739a0a9fc P 4d22b7580880463a9383831743e285e8: This TxnStatusTablet replica's current role is: FOLLOWER
W20260806 18:34:13.116607 21627 tablet.cc:2369] T 2d63df7dd787429089261e8739a0a9fc P ef292515d9c349a8a04b90e9476e31cd: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 18:34:13.118755 21946 raft_consensus.cc:482] T 2d63df7dd787429089261e8739a0a9fc P 8b1a57b1c16a4f71a055f46154de9947 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 18:34:13.118850 21946 raft_consensus.cc:504] T 2d63df7dd787429089261e8739a0a9fc P 8b1a57b1c16a4f71a055f46154de9947 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef292515d9c349a8a04b90e9476e31cd" member_type: VOTER last_known_addr { host: "127.19.194.1" port: 39693 } } peers { permanent_uuid: "4d22b7580880463a9383831743e285e8" member_type: VOTER last_known_addr { host: "127.19.194.2" port: 40521 } } peers { permanent_uuid: "8b1a57b1c16a4f71a055f46154de9947" member_type: VOTER last_known_addr { host: "127.19.194.3" port: 34037 } }
I20260806 18:34:13.119241 21946 leader_election.cc:288] T 2d63df7dd787429089261e8739a0a9fc P 8b1a57b1c16a4f71a055f46154de9947 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ef292515d9c349a8a04b90e9476e31cd (127.19.194.1:39693), 4d22b7580880463a9383831743e285e8 (127.19.194.2:40521)
I20260806 18:34:13.119632 21578 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2d63df7dd787429089261e8739a0a9fc" candidate_uuid: "8b1a57b1c16a4f71a055f46154de9947" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ef292515d9c349a8a04b90e9476e31cd" is_pre_election: true
I20260806 18:34:13.119664 21722 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2d63df7dd787429089261e8739a0a9fc" candidate_uuid: "8b1a57b1c16a4f71a055f46154de9947" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4d22b7580880463a9383831743e285e8" is_pre_election: true
I20260806 18:34:13.119783 21578 raft_consensus.cc:2454] T 2d63df7dd787429089261e8739a0a9fc P ef292515d9c349a8a04b90e9476e31cd [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8b1a57b1c16a4f71a055f46154de9947 in term 0.
I20260806 18:34:13.119815 21722 raft_consensus.cc:2454] T 2d63df7dd787429089261e8739a0a9fc P 4d22b7580880463a9383831743e285e8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8b1a57b1c16a4f71a055f46154de9947 in term 0.
I20260806 18:34:13.120124 21800 leader_election.cc:302] T 2d63df7dd787429089261e8739a0a9fc P 8b1a57b1c16a4f71a055f46154de9947 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8b1a57b1c16a4f71a055f46154de9947, ef292515d9c349a8a04b90e9476e31cd; no voters:
I20260806 18:34:13.120334 21946 raft_consensus.cc:2781] T 2d63df7dd787429089261e8739a0a9fc P 8b1a57b1c16a4f71a055f46154de9947 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 18:34:13.120383 21946 raft_consensus.cc:482] T 2d63df7dd787429089261e8739a0a9fc P 8b1a57b1c16a4f71a055f46154de9947 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 18:34:13.120409 21946 raft_consensus.cc:3037] T 2d63df7dd787429089261e8739a0a9fc P 8b1a57b1c16a4f71a055f46154de9947 [term 0 FOLLOWER]: Advancing to term 1
I20260806 18:34:13.121433 21946 raft_consensus.cc:504] T 2d63df7dd787429089261e8739a0a9fc P 8b1a57b1c16a4f71a055f46154de9947 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef292515d9c349a8a04b90e9476e31cd" member_type: VOTER last_known_addr { host: "127.19.194.1" port: 39693 } } peers { permanent_uuid: "4d22b7580880463a9383831743e285e8" member_type: VOTER last_known_addr { host: "127.19.194.2" port: 40521 } } peers { permanent_uuid: "8b1a57b1c16a4f71a055f46154de9947" member_type: VOTER last_known_addr { host: "127.19.194.3" port: 34037 } }
I20260806 18:34:13.121774 21946 leader_election.cc:288] T 2d63df7dd787429089261e8739a0a9fc P 8b1a57b1c16a4f71a055f46154de9947 [CANDIDATE]: Term 1 election: Requested vote from peers ef292515d9c349a8a04b90e9476e31cd (127.19.194.1:39693), 4d22b7580880463a9383831743e285e8 (127.19.194.2:40521)
I20260806 18:34:13.122068 21578 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2d63df7dd787429089261e8739a0a9fc" candidate_uuid: "8b1a57b1c16a4f71a055f46154de9947" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ef292515d9c349a8a04b90e9476e31cd"
I20260806 18:34:13.122167 21578 raft_consensus.cc:3037] T 2d63df7dd787429089261e8739a0a9fc P ef292515d9c349a8a04b90e9476e31cd [term 0 FOLLOWER]: Advancing to term 1
I20260806 18:34:13.123001 21578 raft_consensus.cc:2454] T 2d63df7dd787429089261e8739a0a9fc P ef292515d9c349a8a04b90e9476e31cd [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8b1a57b1c16a4f71a055f46154de9947 in term 1.
I20260806 18:34:13.123266 21722 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2d63df7dd787429089261e8739a0a9fc" candidate_uuid: "8b1a57b1c16a4f71a055f46154de9947" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4d22b7580880463a9383831743e285e8"
I20260806 18:34:13.123325 21800 leader_election.cc:302] T 2d63df7dd787429089261e8739a0a9fc P 8b1a57b1c16a4f71a055f46154de9947 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8b1a57b1c16a4f71a055f46154de9947, ef292515d9c349a8a04b90e9476e31cd; no voters:
I20260806 18:34:13.123389 21722 raft_consensus.cc:3037] T 2d63df7dd787429089261e8739a0a9fc P 4d22b7580880463a9383831743e285e8 [term 0 FOLLOWER]: Advancing to term 1
I20260806 18:34:13.123490 21946 raft_consensus.cc:2781] T 2d63df7dd787429089261e8739a0a9fc P 8b1a57b1c16a4f71a055f46154de9947 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 18:34:13.123594 21946 raft_consensus.cc:686] T 2d63df7dd787429089261e8739a0a9fc P 8b1a57b1c16a4f71a055f46154de9947 [term 1 LEADER]: Becoming Leader. State: Replica: 8b1a57b1c16a4f71a055f46154de9947, State: Running, Role: LEADER
I20260806 18:34:13.123734 21946 consensus_queue.cc:237] T 2d63df7dd787429089261e8739a0a9fc P 8b1a57b1c16a4f71a055f46154de9947 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef292515d9c349a8a04b90e9476e31cd" member_type: VOTER last_known_addr { host: "127.19.194.1" port: 39693 } } peers { permanent_uuid: "4d22b7580880463a9383831743e285e8" member_type: VOTER last_known_addr { host: "127.19.194.2" port: 40521 } } peers { permanent_uuid: "8b1a57b1c16a4f71a055f46154de9947" member_type: VOTER last_known_addr { host: "127.19.194.3" port: 34037 } }
I20260806 18:34:13.124532 21946 tablet_replica.cc:439] T 2d63df7dd787429089261e8739a0a9fc P 8b1a57b1c16a4f71a055f46154de9947: TxnStatusTablet state changed. Reason: New leader 8b1a57b1c16a4f71a055f46154de9947. Latest consensus state: current_term: 1 leader_uuid: "8b1a57b1c16a4f71a055f46154de9947" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef292515d9c349a8a04b90e9476e31cd" member_type: VOTER last_known_addr { host: "127.19.194.1" port: 39693 } } peers { permanent_uuid: "4d22b7580880463a9383831743e285e8" member_type: VOTER last_known_addr { host: "127.19.194.2" port: 40521 } } peers { permanent_uuid: "8b1a57b1c16a4f71a055f46154de9947" member_type: VOTER last_known_addr { host: "127.19.194.3" port: 34037 } } }
I20260806 18:34:13.124591 21722 raft_consensus.cc:2454] T 2d63df7dd787429089261e8739a0a9fc P 4d22b7580880463a9383831743e285e8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8b1a57b1c16a4f71a055f46154de9947 in term 1.
I20260806 18:34:13.124629 21946 tablet_replica.cc:442] T 2d63df7dd787429089261e8739a0a9fc P 8b1a57b1c16a4f71a055f46154de9947: This TxnStatusTablet replica's current role is: LEADER
I20260806 18:34:13.125422 21979 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260806 18:34:13.125650 21979 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260806 18:34:13.125663 21406 catalog_manager.cc:5495] T 2d63df7dd787429089261e8739a0a9fc P 8b1a57b1c16a4f71a055f46154de9947 reported cstate change: term changed from 0 to 1, leader changed from <none> to 8b1a57b1c16a4f71a055f46154de9947 (127.19.194.3). New cstate: current_term: 1 leader_uuid: "8b1a57b1c16a4f71a055f46154de9947" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef292515d9c349a8a04b90e9476e31cd" member_type: VOTER last_known_addr { host: "127.19.194.1" port: 39693 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4d22b7580880463a9383831743e285e8" member_type: VOTER last_known_addr { host: "127.19.194.2" port: 40521 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8b1a57b1c16a4f71a055f46154de9947" member_type: VOTER last_known_addr { host: "127.19.194.3" port: 34037 } health_report { overall_health: HEALTHY } } }
I20260806 18:34:13.139320 21974 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:34:13.135883 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.194.3:34037 (local address 127.0.0.1:46382)
0806 18:34:13.136053 (+ 170us) negotiation.cc:107] Waiting for socket to connect
0806 18:34:13.136071 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0806 18:34:13.136186 (+ 115us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:34:13.136434 (+ 248us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:34:13.136436 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:34:13.136441 (+ 5us) client_negotiation.cc:189] Negotiated authn=TOKEN
0806 18:34:13.136536 (+ 95us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:34:13.136540 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:34:13.137461 (+ 921us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:34:13.137463 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:34:13.138031 (+ 568us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:34:13.138038 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:34:13.138593 (+ 555us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:34:13.138596 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:34:13.138664 (+ 68us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:34:13.138706 (+ 42us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0806 18:34:13.139027 (+ 321us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0806 18:34:13.139047 (+ 20us) client_negotiation.cc:769] Sending connection context
0806 18:34:13.139099 (+ 52us) client_negotiation.cc:240] Negotiation successful
0806 18:34:13.139222 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":64}
I20260806 18:34:13.139321 21933 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:34:13.135953 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46382 (local address 127.19.194.3:34037)
0806 18:34:13.136073 (+ 120us) server_negotiation.cc:206] Beginning negotiation
0806 18:34:13.136077 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 18:34:13.136125 (+ 48us) server_negotiation.cc:407] Connection header received
0806 18:34:13.136286 (+ 161us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:34:13.136289 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:34:13.136338 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:34:13.136417 (+ 79us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 18:34:13.136633 (+ 216us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:34:13.137367 (+ 734us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:34:13.138131 (+ 764us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:34:13.138488 (+ 357us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:34:13.138587 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:34:13.138834 (+ 247us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 18:34:13.138909 (+ 75us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 18:34:13.139002 (+ 93us) server_negotiation.cc:1035] Waiting for connection context
0806 18:34:13.139145 (+ 143us) server_negotiation.cc:299] Negotiation successful
0806 18:34:13.139197 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260806 18:34:13.141597 21722 raft_consensus.cc:1261] T 2d63df7dd787429089261e8739a0a9fc P 4d22b7580880463a9383831743e285e8 [term 1 FOLLOWER]: Refusing update from remote peer 8b1a57b1c16a4f71a055f46154de9947: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 18:34:13.141597 21578 raft_consensus.cc:1261] T 2d63df7dd787429089261e8739a0a9fc P ef292515d9c349a8a04b90e9476e31cd [term 1 FOLLOWER]: Refusing update from remote peer 8b1a57b1c16a4f71a055f46154de9947: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 18:34:13.142004 21957 consensus_queue.cc:1035] T 2d63df7dd787429089261e8739a0a9fc P 8b1a57b1c16a4f71a055f46154de9947 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4d22b7580880463a9383831743e285e8" member_type: VOTER last_known_addr { host: "127.19.194.2" port: 40521 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 18:34:13.142158 21946 consensus_queue.cc:1035] T 2d63df7dd787429089261e8739a0a9fc P 8b1a57b1c16a4f71a055f46154de9947 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ef292515d9c349a8a04b90e9476e31cd" member_type: VOTER last_known_addr { host: "127.19.194.1" port: 39693 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 18:34:13.144043 21944 tablet_replica.cc:439] T 2d63df7dd787429089261e8739a0a9fc P ef292515d9c349a8a04b90e9476e31cd: TxnStatusTablet state changed. Reason: New leader 8b1a57b1c16a4f71a055f46154de9947. Latest consensus state: current_term: 1 leader_uuid: "8b1a57b1c16a4f71a055f46154de9947" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef292515d9c349a8a04b90e9476e31cd" member_type: VOTER last_known_addr { host: "127.19.194.1" port: 39693 } } peers { permanent_uuid: "4d22b7580880463a9383831743e285e8" member_type: VOTER last_known_addr { host: "127.19.194.2" port: 40521 } } peers { permanent_uuid: "8b1a57b1c16a4f71a055f46154de9947" member_type: VOTER last_known_addr { host: "127.19.194.3" port: 34037 } } }
I20260806 18:34:13.144163 21944 tablet_replica.cc:442] T 2d63df7dd787429089261e8739a0a9fc P ef292515d9c349a8a04b90e9476e31cd: This TxnStatusTablet replica's current role is: FOLLOWER
I20260806 18:34:13.144423 21945 tablet_replica.cc:439] T 2d63df7dd787429089261e8739a0a9fc P 4d22b7580880463a9383831743e285e8: TxnStatusTablet state changed. Reason: New leader 8b1a57b1c16a4f71a055f46154de9947. Latest consensus state: current_term: 1 leader_uuid: "8b1a57b1c16a4f71a055f46154de9947" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef292515d9c349a8a04b90e9476e31cd" member_type: VOTER last_known_addr { host: "127.19.194.1" port: 39693 } } peers { permanent_uuid: "4d22b7580880463a9383831743e285e8" member_type: VOTER last_known_addr { host: "127.19.194.2" port: 40521 } } peers { permanent_uuid: "8b1a57b1c16a4f71a055f46154de9947" member_type: VOTER last_known_addr { host: "127.19.194.3" port: 34037 } } }
I20260806 18:34:13.144539 21945 tablet_replica.cc:442] T 2d63df7dd787429089261e8739a0a9fc P 4d22b7580880463a9383831743e285e8: This TxnStatusTablet replica's current role is: FOLLOWER
I20260806 18:34:13.145390 21949 tablet_replica.cc:439] T 2d63df7dd787429089261e8739a0a9fc P 8b1a57b1c16a4f71a055f46154de9947: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "8b1a57b1c16a4f71a055f46154de9947" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef292515d9c349a8a04b90e9476e31cd" member_type: VOTER last_known_addr { host: "127.19.194.1" port: 39693 } } peers { permanent_uuid: "4d22b7580880463a9383831743e285e8" member_type: VOTER last_known_addr { host: "127.19.194.2" port: 40521 } } peers { permanent_uuid: "8b1a57b1c16a4f71a055f46154de9947" member_type: VOTER last_known_addr { host: "127.19.194.3" port: 34037 } } }
I20260806 18:34:13.145495 21949 tablet_replica.cc:442] T 2d63df7dd787429089261e8739a0a9fc P 8b1a57b1c16a4f71a055f46154de9947: This TxnStatusTablet replica's current role is: LEADER
I20260806 18:34:13.147114 21944 tablet_replica.cc:439] T 2d63df7dd787429089261e8739a0a9fc P ef292515d9c349a8a04b90e9476e31cd: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "8b1a57b1c16a4f71a055f46154de9947" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef292515d9c349a8a04b90e9476e31cd" member_type: VOTER last_known_addr { host: "127.19.194.1" port: 39693 } } peers { permanent_uuid: "4d22b7580880463a9383831743e285e8" member_type: VOTER last_known_addr { host: "127.19.194.2" port: 40521 } } peers { permanent_uuid: "8b1a57b1c16a4f71a055f46154de9947" member_type: VOTER last_known_addr { host: "127.19.194.3" port: 34037 } } }
I20260806 18:34:13.147739 21944 tablet_replica.cc:442] T 2d63df7dd787429089261e8739a0a9fc P ef292515d9c349a8a04b90e9476e31cd: This TxnStatusTablet replica's current role is: FOLLOWER
I20260806 18:34:13.147737 21952 mvcc.cc:205] Tried to move back new op lower bound from 7315624972864376832 to 7315624972796497920. Current Snapshot: MvccSnapshot[applied={T|T < 7315624972864376832 or (T in {7315624972864376832})}]
I20260806 18:34:13.149092 21957 tablet_replica.cc:439] T 2d63df7dd787429089261e8739a0a9fc P 8b1a57b1c16a4f71a055f46154de9947: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "8b1a57b1c16a4f71a055f46154de9947" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef292515d9c349a8a04b90e9476e31cd" member_type: VOTER last_known_addr { host: "127.19.194.1" port: 39693 } } peers { permanent_uuid: "4d22b7580880463a9383831743e285e8" member_type: VOTER last_known_addr { host: "127.19.194.2" port: 40521 } } peers { permanent_uuid: "8b1a57b1c16a4f71a055f46154de9947" member_type: VOTER last_known_addr { host: "127.19.194.3" port: 34037 } } }
I20260806 18:34:13.149197 21957 tablet_replica.cc:442] T 2d63df7dd787429089261e8739a0a9fc P 8b1a57b1c16a4f71a055f46154de9947: This TxnStatusTablet replica's current role is: LEADER
I20260806 18:34:13.149720 21954 mvcc.cc:205] Tried to move back new op lower bound from 7315624972864376832 to 7315624972796497920. Current Snapshot: MvccSnapshot[applied={T|T < 7315624972864376832}]
I20260806 18:34:13.149895 21945 tablet_replica.cc:439] T 2d63df7dd787429089261e8739a0a9fc P 4d22b7580880463a9383831743e285e8: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "8b1a57b1c16a4f71a055f46154de9947" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef292515d9c349a8a04b90e9476e31cd" member_type: VOTER last_known_addr { host: "127.19.194.1" port: 39693 } } peers { permanent_uuid: "4d22b7580880463a9383831743e285e8" member_type: VOTER last_known_addr { host: "127.19.194.2" port: 40521 } } peers { permanent_uuid: "8b1a57b1c16a4f71a055f46154de9947" member_type: VOTER last_known_addr { host: "127.19.194.3" port: 34037 } } }
I20260806 18:34:13.149981 21945 tablet_replica.cc:442] T 2d63df7dd787429089261e8739a0a9fc P 4d22b7580880463a9383831743e285e8: This TxnStatusTablet replica's current role is: FOLLOWER
I20260806 18:34:13.159864 21933 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:34:13.156745 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.194.3:58821 (local address 127.19.194.3:34037)
0806 18:34:13.156847 (+ 102us) server_negotiation.cc:206] Beginning negotiation
0806 18:34:13.156856 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0806 18:34:13.156869 (+ 13us) server_negotiation.cc:407] Connection header received
0806 18:34:13.156980 (+ 111us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:34:13.156982 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:34:13.157021 (+ 39us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:34:13.157084 (+ 63us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 18:34:13.157331 (+ 247us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:34:13.157787 (+ 456us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:34:13.158899 (+ 1112us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:34:13.159090 (+ 191us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:34:13.159237 (+ 147us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:34:13.159519 (+ 282us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 18:34:13.159582 (+ 63us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 18:34:13.159636 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0806 18:34:13.159737 (+ 101us) server_negotiation.cc:299] Negotiation successful
0806 18:34:13.159773 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260806 18:34:13.159965 21920 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:34:13.156672 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.194.3:34037 (local address 127.19.194.3:58821)
0806 18:34:13.156810 (+ 138us) negotiation.cc:107] Waiting for socket to connect
0806 18:34:13.156826 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0806 18:34:13.156901 (+ 75us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:34:13.157116 (+ 215us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:34:13.157119 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:34:13.157126 (+ 7us) client_negotiation.cc:189] Negotiated authn=TOKEN
0806 18:34:13.157231 (+ 105us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:34:13.157236 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:34:13.157880 (+ 644us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:34:13.157884 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:34:13.158794 (+ 910us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:34:13.158804 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:34:13.159256 (+ 452us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:34:13.159260 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:34:13.159343 (+ 83us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:34:13.159393 (+ 50us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0806 18:34:13.159658 (+ 265us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0806 18:34:13.159665 (+ 7us) client_negotiation.cc:769] Sending connection context
0806 18:34:13.159733 (+ 68us) client_negotiation.cc:240] Negotiation successful
0806 18:34:13.159810 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":30}
I20260806 18:34:13.166674 21722 raft_consensus.cc:1261] T cfa8797235e94ae9be65666f05e8f629 P 4d22b7580880463a9383831743e285e8 [term 1 FOLLOWER]: Refusing update from remote peer 8b1a57b1c16a4f71a055f46154de9947: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 18:34:13.166792 21578 raft_consensus.cc:1261] T cfa8797235e94ae9be65666f05e8f629 P ef292515d9c349a8a04b90e9476e31cd [term 1 FOLLOWER]: Refusing update from remote peer 8b1a57b1c16a4f71a055f46154de9947: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 18:34:13.167049 21946 consensus_queue.cc:1035] T cfa8797235e94ae9be65666f05e8f629 P 8b1a57b1c16a4f71a055f46154de9947 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4d22b7580880463a9383831743e285e8" member_type: VOTER last_known_addr { host: "127.19.194.2" port: 40521 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 18:34:13.167212 21957 consensus_queue.cc:1035] T cfa8797235e94ae9be65666f05e8f629 P 8b1a57b1c16a4f71a055f46154de9947 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ef292515d9c349a8a04b90e9476e31cd" member_type: VOTER last_known_addr { host: "127.19.194.1" port: 39693 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 18:34:13.207551 21406 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:47880:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 18:34:13.207724 21406 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:47880:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 18:34:13.209728 21406 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 4bfe62b1286a4f54925c8a7b156a6adf: Sending DeleteTablet for 3 replicas of tablet cfa8797235e94ae9be65666f05e8f629
I20260806 18:34:13.210219 21556 tablet_service.cc:1514] Processing DeleteTablet for tablet cfa8797235e94ae9be65666f05e8f629 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 18:34:13 UTC) from {username='kudu', principal='kudu/127.19.194.62@KRBTEST.COM'} at 127.0.0.1:55038
I20260806 18:34:13.210412 21963 tablet_replica.cc:330] T cfa8797235e94ae9be65666f05e8f629 P ef292515d9c349a8a04b90e9476e31cd: stopping tablet replica
I20260806 18:34:13.210429 21847 tablet_service.cc:1514] Processing DeleteTablet for tablet cfa8797235e94ae9be65666f05e8f629 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 18:34:13 UTC) from {username='kudu', principal='kudu/127.19.194.62@KRBTEST.COM'} at 127.0.0.1:46366
I20260806 18:34:13.210559 21963 raft_consensus.cc:2229] T cfa8797235e94ae9be65666f05e8f629 P ef292515d9c349a8a04b90e9476e31cd [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 18:34:13.210616 21962 tablet_replica.cc:330] T cfa8797235e94ae9be65666f05e8f629 P 8b1a57b1c16a4f71a055f46154de9947: stopping tablet replica
I20260806 18:34:13.210711 21963 raft_consensus.cc:2258] T cfa8797235e94ae9be65666f05e8f629 P ef292515d9c349a8a04b90e9476e31cd [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 18:34:13.210743 21962 raft_consensus.cc:2229] T cfa8797235e94ae9be65666f05e8f629 P 8b1a57b1c16a4f71a055f46154de9947 [term 1 LEADER]: Raft consensus shutting down.
I20260806 18:34:13.210876 21702 tablet_service.cc:1514] Processing DeleteTablet for tablet cfa8797235e94ae9be65666f05e8f629 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 18:34:13 UTC) from {username='kudu', principal='kudu/127.19.194.62@KRBTEST.COM'} at 127.0.0.1:47116
I20260806 18:34:13.210906 21962 raft_consensus.cc:2258] T cfa8797235e94ae9be65666f05e8f629 P 8b1a57b1c16a4f71a055f46154de9947 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 18:34:13.211210 21964 tablet_replica.cc:330] T cfa8797235e94ae9be65666f05e8f629 P 4d22b7580880463a9383831743e285e8: stopping tablet replica
I20260806 18:34:13.211334 21964 raft_consensus.cc:2229] T cfa8797235e94ae9be65666f05e8f629 P 4d22b7580880463a9383831743e285e8 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 18:34:13.211436 21964 raft_consensus.cc:2258] T cfa8797235e94ae9be65666f05e8f629 P 4d22b7580880463a9383831743e285e8 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 18:34:13.211650 21963 ts_tablet_manager.cc:1905] T cfa8797235e94ae9be65666f05e8f629 P ef292515d9c349a8a04b90e9476e31cd: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 18:34:13.212182 21962 ts_tablet_manager.cc:1905] T cfa8797235e94ae9be65666f05e8f629 P 8b1a57b1c16a4f71a055f46154de9947: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 18:34:13.214629 21963 ts_tablet_manager.cc:1918] T cfa8797235e94ae9be65666f05e8f629 P ef292515d9c349a8a04b90e9476e31cd: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260806 18:34:13.214706 21963 log.cc:1196] T cfa8797235e94ae9be65666f05e8f629 P ef292515d9c349a8a04b90e9476e31cd: Deleting WAL directory at /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786041246094603-20232-0/minicluster-data/ts-0/wal/wals/cfa8797235e94ae9be65666f05e8f629
I20260806 18:34:13.214901 21962 ts_tablet_manager.cc:1918] T cfa8797235e94ae9be65666f05e8f629 P 8b1a57b1c16a4f71a055f46154de9947: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260806 18:34:13.214972 21962 log.cc:1196] T cfa8797235e94ae9be65666f05e8f629 P 8b1a57b1c16a4f71a055f46154de9947: Deleting WAL directory at /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786041246094603-20232-0/minicluster-data/ts-2/wal/wals/cfa8797235e94ae9be65666f05e8f629
I20260806 18:34:13.214973 21963 ts_tablet_manager.cc:1939] T cfa8797235e94ae9be65666f05e8f629 P ef292515d9c349a8a04b90e9476e31cd: Deleting consensus metadata
I20260806 18:34:13.215241 21962 ts_tablet_manager.cc:1939] T cfa8797235e94ae9be65666f05e8f629 P 8b1a57b1c16a4f71a055f46154de9947: Deleting consensus metadata
I20260806 18:34:13.215687 21390 catalog_manager.cc:4841] TS ef292515d9c349a8a04b90e9476e31cd (127.19.194.1:39693): tablet cfa8797235e94ae9be65666f05e8f629 (table test-table [id=cbff0f0c226f4de3b11d42281d252c60]) successfully deleted
I20260806 18:34:13.215943 21391 catalog_manager.cc:4841] TS 8b1a57b1c16a4f71a055f46154de9947 (127.19.194.3:34037): tablet cfa8797235e94ae9be65666f05e8f629 (table test-table [id=cbff0f0c226f4de3b11d42281d252c60]) successfully deleted
I20260806 18:34:13.216957 21964 ts_tablet_manager.cc:1905] T cfa8797235e94ae9be65666f05e8f629 P 4d22b7580880463a9383831743e285e8: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 18:34:13.218947 21964 ts_tablet_manager.cc:1918] T cfa8797235e94ae9be65666f05e8f629 P 4d22b7580880463a9383831743e285e8: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260806 18:34:13.219049 21964 log.cc:1196] T cfa8797235e94ae9be65666f05e8f629 P 4d22b7580880463a9383831743e285e8: Deleting WAL directory at /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786041246094603-20232-0/minicluster-data/ts-1/wal/wals/cfa8797235e94ae9be65666f05e8f629
I20260806 18:34:13.219379 21964 ts_tablet_manager.cc:1939] T cfa8797235e94ae9be65666f05e8f629 P 4d22b7580880463a9383831743e285e8: Deleting consensus metadata
I20260806 18:34:13.220001 21391 catalog_manager.cc:4841] TS 4d22b7580880463a9383831743e285e8 (127.19.194.2:40521): tablet cfa8797235e94ae9be65666f05e8f629 (table test-table [id=cbff0f0c226f4de3b11d42281d252c60]) successfully deleted
Aug 06 18:34:13 dist-test-slave-1xrh krb5kdc[21356](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041252, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.19.194.1@KRBTEST.COM
I20260806 18:34:13.231314 21934 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:34:13.219213 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55046 (local address 127.19.194.1:39693)
0806 18:34:13.219346 (+ 133us) server_negotiation.cc:206] Beginning negotiation
0806 18:34:13.219351 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 18:34:13.219367 (+ 16us) server_negotiation.cc:407] Connection header received
0806 18:34:13.219407 (+ 40us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:34:13.219413 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:34:13.219467 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:34:13.219571 (+ 104us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:34:13.220480 (+ 909us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:34:13.221121 (+ 641us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:34:13.222304 (+ 1183us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:34:13.222493 (+ 189us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:34:13.222575 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:34:13.227579 (+ 5004us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:34:13.227590 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:34:13.227603 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 18:34:13.227625 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:34:13.229504 (+ 1879us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:34:13.229919 (+ 415us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:34:13.229926 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:34:13.229928 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:34:13.230022 (+ 94us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:34:13.230293 (+ 271us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:34:13.230299 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:34:13.230302 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:34:13.230561 (+ 259us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:34:13.230815 (+ 254us) server_negotiation.cc:1035] Waiting for connection context
0806 18:34:13.230977 (+ 162us) server_negotiation.cc:299] Negotiation successful
0806 18:34:13.231103 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34}
W20260806 18:34:13.231717 21619 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.SetFlag from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:55046
I20260806 18:34:13.240924 21460 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:34:13.233370 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47902 (local address 127.19.194.62:34683)
0806 18:34:13.233492 (+ 122us) server_negotiation.cc:206] Beginning negotiation
0806 18:34:13.233496 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 18:34:13.233536 (+ 40us) server_negotiation.cc:407] Connection header received
0806 18:34:13.233661 (+ 125us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:34:13.233663 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:34:13.233700 (+ 37us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:34:13.233761 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:34:13.234537 (+ 776us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:34:13.235055 (+ 518us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:34:13.236243 (+ 1188us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:34:13.236454 (+ 211us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:34:13.236567 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:34:13.237946 (+ 1379us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:34:13.237949 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:34:13.237951 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 18:34:13.237973 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:34:13.239478 (+ 1505us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:34:13.239917 (+ 439us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:34:13.239920 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:34:13.239921 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:34:13.239993 (+ 72us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:34:13.240242 (+ 249us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:34:13.240244 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:34:13.240245 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:34:13.240398 (+ 153us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:34:13.240493 (+ 95us) server_negotiation.cc:1035] Waiting for connection context
0806 18:34:13.240699 (+ 206us) server_negotiation.cc:299] Negotiation successful
0806 18:34:13.240785 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
W20260806 18:34:13.241321 21406 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.TSHeartbeat from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:47902
I20260806 18:34:13.242823 20232 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJW69tn/build/debug/bin/kudu with pid 21487
I20260806 18:34:13.252486 20232 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJW69tn/build/debug/bin/kudu with pid 21635
I20260806 18:34:13.259752 20232 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJW69tn/build/debug/bin/kudu with pid 21781
I20260806 18:34:13.267761 20232 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJW69tn/build/debug/bin/kudu with pid 21370
2026-08-06T18:34:13Z chronyd exiting
[ OK ] SecurityITest.SmokeTestAsAuthorizedUser (2545 ms)
[ RUN ] SecurityITest.TxnSmokeWithDifferentUserTypes
Loading random data
Initializing database '/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786041246094603-20232-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 18:34:13 dist-test-slave-1xrh krb5kdc[21994](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 18:34:13 dist-test-slave-1xrh krb5kdc[21994](info): set up 2 sockets
Aug 06 18:34:13 dist-test-slave-1xrh krb5kdc[21994](info): commencing operation
krb5kdc: starting...
W20260806 18:34:14.324347 20232 mini_kdc.cc:121] Time spent starting KDC: real 1.033s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786041246094603-20232-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786041246094603-20232-0/krb5kdc/test-user.keytab.
Aug 06 18:34:14 dist-test-slave-1xrh krb5kdc[21994](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041254, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T18:34:14Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T18:34:14Z Disabled control of system clock
WARNING: no policy specified for kudu/127.19.194.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.19.194.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.19.194.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786041246094603-20232-0/krb5kdc/kudu_127.19.194.62.keytab.
Entry for principal kudu/127.19.194.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786041246094603-20232-0/krb5kdc/kudu_127.19.194.62.keytab.
I20260806 18:34:14.441701 20232 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJW69tn/build/debug/bin/kudu
/tmp/dist-test-taskJW69tn/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786041246094603-20232-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786041246094603-20232-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786041246094603-20232-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786041246094603-20232-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.19.194.62:33617
--webserver_interface=127.19.194.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786041246094603-20232-0/krb5kdc/kudu_127.19.194.62.keytab
--principal=kudu/127.19.194.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.19.194.20:42515
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.19.194.62:33617
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786041246094603-20232-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786041246094603-20232-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786041246094603-20232-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 18:34:14.549404 22007 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 18:34:14.549630 22007 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 18:34:14.549667 22007 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 18:34:14.553599 22007 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 18:34:14.553653 22007 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260806 18:34:14.553669 22007 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 18:34:14.553687 22007 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 18:34:14.553721 22007 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 18:34:14.553757 22007 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 18:34:14.558887 22007 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.194.20:42515
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786041246094603-20232-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786041246094603-20232-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.19.194.62:33617
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786041246094603-20232-0/krb5kdc/kudu_127.19.194.62.keytab
--principal=kudu/127.19.194.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.19.194.62:33617
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786041246094603-20232-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.19.194.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22007
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786041246094603-20232-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 17:59:13 UTC on ccycloud.quasar-wvsnzx.root.comops.site
build id 4571028
I20260806 18:34:14.559862 22007 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 18:34:14.560812 22007 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 18:34:14.568849 22015 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:34:14.568846 22012 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:34:14.568889 22013 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 18:34:14.569182 22007 server_base.cc:1034] running on GCE node
I20260806 18:34:14.569758 22007 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 18:34:14.570775 22007 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 18:34:14.571933 22007 hybrid_clock.cc:648] HybridClock initialized: now 1786041254571904 us; error 42 us; skew 500 ppm
Aug 06 18:34:14 dist-test-slave-1xrh krb5kdc[21994](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041254, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.194.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 18:34:14.575062 22007 init.cc:377] Logged in from keytab as kudu/127.19.194.62@KRBTEST.COM (short username kudu)
I20260806 18:34:14.575953 22007 webserver.cc:460] Webserver started at http://127.19.194.62:35073/ using document root <none> and password file <none>
I20260806 18:34:14.576491 22007 fs_manager.cc:359] Metadata directory not provided
I20260806 18:34:14.576553 22007 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 18:34:14.576776 22007 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 18:34:14.578714 22007 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786041246094603-20232-0/minicluster-data/master-0/data/instance:
uuid: "805a3be867fa43bdafe4c8974a803d51"
format_stamp: "Formatted at 2026-08-06 18:34:14 on dist-test-slave-1xrh"
server_key: "d0187b7080daf17794f6f7b674f4654c"
server_key_iv: "dccc993a1ceb15b62defb8b8663747dc"
server_key_version: "encryptionkey@0"
I20260806 18:34:14.579324 22007 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786041246094603-20232-0/minicluster-data/master-0/wal/instance:
uuid: "805a3be867fa43bdafe4c8974a803d51"
format_stamp: "Formatted at 2026-08-06 18:34:14 on dist-test-slave-1xrh"
server_key: "d0187b7080daf17794f6f7b674f4654c"
server_key_iv: "dccc993a1ceb15b62defb8b8663747dc"
server_key_version: "encryptionkey@0"
I20260806 18:34:14.582814 22007 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.000s
I20260806 18:34:14.585152 22022 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 18:34:14.586326 22007 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260806 18:34:14.586457 22007 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786041246094603-20232-0/minicluster-data/master-0/data,/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786041246094603-20232-0/minicluster-data/master-0/wal
uuid: "805a3be867fa43bdafe4c8974a803d51"
format_stamp: "Formatted at 2026-08-06 18:34:14 on dist-test-slave-1xrh"
server_key: "d0187b7080daf17794f6f7b674f4654c"
server_key_iv: "dccc993a1ceb15b62defb8b8663747dc"
server_key_version: "encryptionkey@0"
I20260806 18:34:14.586571 22007 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786041246094603-20232-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786041246094603-20232-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786041246094603-20232-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 18:34:14.595338 22007 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 18:34:14.598306 22007 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 18:34:14.598506 22007 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 18:34:14.606307 22007 rpc_server.cc:307] RPC server started. Bound to: 127.19.194.62:33617
I20260806 18:34:14.606335 22086 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.194.62:33617 every 8 connection(s)
I20260806 18:34:14.607204 22007 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786041246094603-20232-0/minicluster-data/master-0/data/info.pb
I20260806 18:34:14.609833 22087 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 18:34:14.615381 22087 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 805a3be867fa43bdafe4c8974a803d51: Bootstrap starting.
I20260806 18:34:14.617136 20232 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJW69tn/build/debug/bin/kudu as pid 22007
I20260806 18:34:14.617252 20232 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786041246094603-20232-0/minicluster-data/master-0/wal/instance
I20260806 18:34:14.617511 20232 external_mini_cluster.cc:1428] Setting key fa32515aaaf0db5dbedcdd9c5ede4f66
I20260806 18:34:14.618059 22087 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 805a3be867fa43bdafe4c8974a803d51: Neither blocks nor log segments found. Creating new log.
I20260806 18:34:14.618954 22087 log.cc:824] T 00000000000000000000000000000000 P 805a3be867fa43bdafe4c8974a803d51: Log is configured to *not* fsync() on all Append() calls
I20260806 18:34:14.620823 22087 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 805a3be867fa43bdafe4c8974a803d51: No bootstrap required, opened a new log
I20260806 18:34:14.624191 22087 raft_consensus.cc:348] T 00000000000000000000000000000000 P 805a3be867fa43bdafe4c8974a803d51 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "805a3be867fa43bdafe4c8974a803d51" member_type: VOTER last_known_addr { host: "127.19.194.62" port: 33617 } }
I20260806 18:34:14.624439 22087 raft_consensus.cc:374] T 00000000000000000000000000000000 P 805a3be867fa43bdafe4c8974a803d51 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 18:34:14.624512 22087 raft_consensus.cc:729] T 00000000000000000000000000000000 P 805a3be867fa43bdafe4c8974a803d51 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 805a3be867fa43bdafe4c8974a803d51, State: Initialized, Role: FOLLOWER
I20260806 18:34:14.624912 22087 consensus_queue.cc:260] T 00000000000000000000000000000000 P 805a3be867fa43bdafe4c8974a803d51 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "805a3be867fa43bdafe4c8974a803d51" member_type: VOTER last_known_addr { host: "127.19.194.62" port: 33617 } }
I20260806 18:34:14.625053 22087 raft_consensus.cc:388] T 00000000000000000000000000000000 P 805a3be867fa43bdafe4c8974a803d51 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 18:34:14.625134 22087 raft_consensus.cc:482] T 00000000000000000000000000000000 P 805a3be867fa43bdafe4c8974a803d51 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 18:34:14.625247 22087 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 805a3be867fa43bdafe4c8974a803d51 [term 0 FOLLOWER]: Advancing to term 1
Aug 06 18:34:14 dist-test-slave-1xrh krb5kdc[21994](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041254, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.19.194.62@KRBTEST.COM
I20260806 18:34:14.626346 22087 raft_consensus.cc:504] T 00000000000000000000000000000000 P 805a3be867fa43bdafe4c8974a803d51 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "805a3be867fa43bdafe4c8974a803d51" member_type: VOTER last_known_addr { host: "127.19.194.62" port: 33617 } }
I20260806 18:34:14.626713 22087 leader_election.cc:302] T 00000000000000000000000000000000 P 805a3be867fa43bdafe4c8974a803d51 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 805a3be867fa43bdafe4c8974a803d51; no voters:
I20260806 18:34:14.627030 22087 leader_election.cc:288] T 00000000000000000000000000000000 P 805a3be867fa43bdafe4c8974a803d51 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 18:34:14.627197 22092 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 805a3be867fa43bdafe4c8974a803d51 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 18:34:14.627502 22092 raft_consensus.cc:686] T 00000000000000000000000000000000 P 805a3be867fa43bdafe4c8974a803d51 [term 1 LEADER]: Becoming Leader. State: Replica: 805a3be867fa43bdafe4c8974a803d51, State: Running, Role: LEADER
I20260806 18:34:14.627873 22092 consensus_queue.cc:237] T 00000000000000000000000000000000 P 805a3be867fa43bdafe4c8974a803d51 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "805a3be867fa43bdafe4c8974a803d51" member_type: VOTER last_known_addr { host: "127.19.194.62" port: 33617 } }
I20260806 18:34:14.628340 22087 sys_catalog.cc:564] T 00000000000000000000000000000000 P 805a3be867fa43bdafe4c8974a803d51 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 18:34:14.630152 22093 sys_catalog.cc:455] T 00000000000000000000000000000000 P 805a3be867fa43bdafe4c8974a803d51 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 805a3be867fa43bdafe4c8974a803d51. Latest consensus state: current_term: 1 leader_uuid: "805a3be867fa43bdafe4c8974a803d51" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "805a3be867fa43bdafe4c8974a803d51" member_type: VOTER last_known_addr { host: "127.19.194.62" port: 33617 } } }
I20260806 18:34:14.630277 22093 sys_catalog.cc:458] T 00000000000000000000000000000000 P 805a3be867fa43bdafe4c8974a803d51 [sys.catalog]: This master's current role is: LEADER
I20260806 18:34:14.630020 22094 sys_catalog.cc:455] T 00000000000000000000000000000000 P 805a3be867fa43bdafe4c8974a803d51 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "805a3be867fa43bdafe4c8974a803d51" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "805a3be867fa43bdafe4c8974a803d51" member_type: VOTER last_known_addr { host: "127.19.194.62" port: 33617 } } }
I20260806 18:34:14.631052 22094 sys_catalog.cc:458] T 00000000000000000000000000000000 P 805a3be867fa43bdafe4c8974a803d51 [sys.catalog]: This master's current role is: LEADER
I20260806 18:34:14.631347 22107 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 18:34:14.632539 22090 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:34:14.618595 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40684 (local address 127.19.194.62:33617)
0806 18:34:14.619037 (+ 442us) server_negotiation.cc:206] Beginning negotiation
0806 18:34:14.619051 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0806 18:34:14.619128 (+ 77us) server_negotiation.cc:407] Connection header received
0806 18:34:14.619824 (+ 696us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:34:14.619840 (+ 16us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:34:14.620147 (+ 307us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:34:14.620461 (+ 314us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:34:14.621312 (+ 851us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:34:14.622257 (+ 945us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:34:14.623145 (+ 888us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:34:14.623611 (+ 466us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:34:14.623794 (+ 183us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:34:14.626724 (+ 2930us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:34:14.626729 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:34:14.626741 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 18:34:14.626785 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:34:14.628889 (+ 2104us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:34:14.629437 (+ 548us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:34:14.629444 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:34:14.629449 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:34:14.629599 (+ 150us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:34:14.629996 (+ 397us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:34:14.630000 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:34:14.630002 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:34:14.630557 (+ 555us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:34:14.630744 (+ 187us) server_negotiation.cc:1035] Waiting for connection context
0806 18:34:14.631446 (+ 702us) server_negotiation.cc:299] Negotiation successful
0806 18:34:14.631759 (+ 313us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":306,"thread_start_us":139,"threads_started":1}
I20260806 18:34:14.634426 22107 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 18:34:14.639652 22107 catalog_manager.cc:1347] Generated new cluster ID: 7003d7f3ee394fa29385e450bc49a8f4
I20260806 18:34:14.639712 22107 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 18:34:14.651253 22107 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 18:34:14.652209 22107 catalog_manager.cc:1504] Loading token signing keys...
I20260806 18:34:14.656690 22107 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 805a3be867fa43bdafe4c8974a803d51: Generated new TSK 0
I20260806 18:34:14.657294 22107 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.19.194.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.19.194.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.19.194.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786041246094603-20232-0/krb5kdc/kudu_127.19.194.1.keytab.
Entry for principal kudu/127.19.194.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786041246094603-20232-0/krb5kdc/kudu_127.19.194.1.keytab.
I20260806 18:34:14.694427 20232 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJW69tn/build/debug/bin/kudu
/tmp/dist-test-taskJW69tn/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786041246094603-20232-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786041246094603-20232-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786041246094603-20232-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786041246094603-20232-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.19.194.1:0
--local_ip_for_outbound_sockets=127.19.194.1
--webserver_interface=127.19.194.1
--webserver_port=0
--tserver_master_addrs=127.19.194.62:33617
--keytab_file=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786041246094603-20232-0/krb5kdc/kudu_127.19.194.1.keytab
--principal=kudu/127.19.194.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.19.194.20:42515
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786041246094603-20232-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786041246094603-20232-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786041246094603-20232-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 18:34:14.804199 22113 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 18:34:14.804427 22113 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 18:34:14.804472 22113 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 18:34:14.808205 22113 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 18:34:14.808250 22113 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 18:34:14.808286 22113 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260806 18:34:14.808300 22113 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260806 18:34:14.808332 22113 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.19.194.1
I20260806 18:34:14.813227 22113 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.194.20:42515
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786041246094603-20232-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786041246094603-20232-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786041246094603-20232-0/krb5kdc/kudu_127.19.194.1.keytab
--principal=kudu/127.19.194.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.19.194.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786041246094603-20232-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.19.194.1
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.19.194.62:33617
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22113
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.19.194.1
--log_dir=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786041246094603-20232-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 17:59:13 UTC on ccycloud.quasar-wvsnzx.root.comops.site
build id 4571028
I20260806 18:34:14.814153 22113 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 18:34:14.815071 22113 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 18:34:14.824230 22118 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:34:14.824283 22119 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 18:34:14.824292 22113 server_base.cc:1034] running on GCE node
W20260806 18:34:14.824244 22121 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 18:34:14.824808 22113 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 18:34:14.825418 22113 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 18:34:14.826577 22113 hybrid_clock.cc:648] HybridClock initialized: now 1786041254826544 us; error 45 us; skew 500 ppm
Aug 06 18:34:14 dist-test-slave-1xrh krb5kdc[21994](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041254, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.194.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 18:34:14.829324 22113 init.cc:377] Logged in from keytab as kudu/127.19.194.1@KRBTEST.COM (short username kudu)
I20260806 18:34:14.830256 22113 webserver.cc:460] Webserver started at http://127.19.194.1:39589/ using document root <none> and password file <none>
I20260806 18:34:14.830808 22113 fs_manager.cc:359] Metadata directory not provided
I20260806 18:34:14.830894 22113 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 18:34:14.831107 22113 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 18:34:14.833005 22113 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786041246094603-20232-0/minicluster-data/ts-0/data/instance:
uuid: "39d31d4b04c14099bf89458586ff793c"
format_stamp: "Formatted at 2026-08-06 18:34:14 on dist-test-slave-1xrh"
server_key: "0c4ae384dd320c3dbe66d7c69995c5aa"
server_key_iv: "4f4e985946f3f4f5e556daf6313ddf4e"
server_key_version: "encryptionkey@0"
I20260806 18:34:14.833575 22113 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786041246094603-20232-0/minicluster-data/ts-0/wal/instance:
uuid: "39d31d4b04c14099bf89458586ff793c"
format_stamp: "Formatted at 2026-08-06 18:34:14 on dist-test-slave-1xrh"
server_key: "0c4ae384dd320c3dbe66d7c69995c5aa"
server_key_iv: "4f4e985946f3f4f5e556daf6313ddf4e"
server_key_version: "encryptionkey@0"
I20260806 18:34:14.837067 22113 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.005s sys 0.000s
I20260806 18:34:14.839295 22128 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 18:34:14.840447 22113 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260806 18:34:14.840582 22113 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786041246094603-20232-0/minicluster-data/ts-0/data,/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786041246094603-20232-0/minicluster-data/ts-0/wal
uuid: "39d31d4b04c14099bf89458586ff793c"
format_stamp: "Formatted at 2026-08-06 18:34:14 on dist-test-slave-1xrh"
server_key: "0c4ae384dd320c3dbe66d7c69995c5aa"
server_key_iv: "4f4e985946f3f4f5e556daf6313ddf4e"
server_key_version: "encryptionkey@0"
I20260806 18:34:14.840688 22113 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786041246094603-20232-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786041246094603-20232-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786041246094603-20232-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 18:34:14.860560 22113 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 18:34:14.862963 22113 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 18:34:14.863196 22113 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 18:34:14.864522 22113 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 18:34:14.864591 22113 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 18:34:14.864656 22113 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 18:34:14.864703 22113 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 06 18:34:14 dist-test-slave-1xrh krb5kdc[21994](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041254, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.194.1@KRBTEST.COM for kudu/127.19.194.62@KRBTEST.COM
I20260806 18:34:14.875345 22113 rpc_server.cc:307] RPC server started. Bound to: 127.19.194.1:37593
I20260806 18:34:14.875599 22247 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.194.1:37593 every 8 connection(s)
I20260806 18:34:14.876245 22113 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786041246094603-20232-0/minicluster-data/ts-0/data/info.pb
I20260806 18:34:14.878737 22090 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:34:14.865366 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.194.1:38447 (local address 127.19.194.62:33617)
0806 18:34:14.865483 (+ 117us) server_negotiation.cc:206] Beginning negotiation
0806 18:34:14.865487 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 18:34:14.866455 (+ 968us) server_negotiation.cc:407] Connection header received
0806 18:34:14.867487 (+ 1032us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:34:14.867491 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:34:14.867551 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:34:14.867615 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:34:14.869344 (+ 1729us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:34:14.870323 (+ 979us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:34:14.872111 (+ 1788us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:34:14.872518 (+ 407us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:34:14.872640 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:34:14.875521 (+ 2881us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:34:14.875537 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:34:14.875540 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 18:34:14.875573 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:34:14.877172 (+ 1599us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:34:14.877590 (+ 418us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:34:14.877592 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:34:14.877593 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:34:14.877662 (+ 69us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:34:14.877923 (+ 261us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:34:14.877926 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:34:14.877927 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:34:14.878081 (+ 154us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:34:14.878148 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0806 18:34:14.878529 (+ 381us) server_negotiation.cc:299] Negotiation successful
0806 18:34:14.878626 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260806 18:34:14.879436 22137 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:34:14.865857 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.194.62:33617 (local address 127.19.194.1:38447)
0806 18:34:14.866314 (+ 457us) negotiation.cc:107] Waiting for socket to connect
0806 18:34:14.866352 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0806 18:34:14.867242 (+ 890us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:34:14.868109 (+ 867us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:34:14.868120 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:34:14.868603 (+ 483us) client_negotiation.cc:189] Negotiated authn=SASL
0806 18:34:14.869184 (+ 581us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:34:14.869207 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:34:14.870466 (+ 1259us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:34:14.870472 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:34:14.871952 (+ 1480us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:34:14.871965 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:34:14.872677 (+ 712us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:34:14.872680 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:34:14.872816 (+ 136us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:34:14.873387 (+ 571us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 18:34:14.873413 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0806 18:34:14.875326 (+ 1913us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 18:34:14.877266 (+ 1940us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:34:14.877270 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:34:14.877279 (+ 9us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:34:14.877503 (+ 224us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:34:14.877730 (+ 227us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:34:14.877732 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:34:14.877733 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:34:14.877848 (+ 115us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:34:14.878159 (+ 311us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 18:34:14.878162 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 18:34:14.878342 (+ 180us) client_negotiation.cc:769] Sending connection context
0806 18:34:14.878471 (+ 129us) client_negotiation.cc:240] Negotiation successful
0806 18:34:14.878674 (+ 203us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":285,"thread_start_us":129,"threads_started":1}
I20260806 18:34:14.880494 20232 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJW69tn/build/debug/bin/kudu as pid 22113
I20260806 18:34:14.880627 20232 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786041246094603-20232-0/minicluster-data/ts-0/wal/instance
I20260806 18:34:14.880716 22248 heartbeater.cc:347] Connected to a master server at 127.19.194.62:33617
I20260806 18:34:14.880846 22248 heartbeater.cc:464] Registering TS with master...
I20260806 18:34:14.880934 20232 external_mini_cluster.cc:1428] Setting key 2660c9aef7182617944cfdecb3bfef80
I20260806 18:34:14.881389 22248 heartbeater.cc:511] Master 127.19.194.62:33617 requested a full tablet report, sending...
I20260806 18:34:14.882937 22038 ts_manager.cc:194] Registered new tserver with Master: 39d31d4b04c14099bf89458586ff793c (127.19.194.1:37593)
I20260806 18:34:14.884444 22038 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.19.194.1@KRBTEST.COM'} at 127.19.194.1:38447
I20260806 18:34:14.890700 22090 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:34:14.882763 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.194.1:53013 (local address 127.19.194.62:33617)
0806 18:34:14.882864 (+ 101us) server_negotiation.cc:206] Beginning negotiation
0806 18:34:14.882867 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 18:34:14.883143 (+ 276us) server_negotiation.cc:407] Connection header received
0806 18:34:14.883253 (+ 110us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:34:14.883259 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:34:14.883314 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:34:14.883398 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:34:14.883943 (+ 545us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:34:14.884686 (+ 743us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:34:14.885699 (+ 1013us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:34:14.886091 (+ 392us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:34:14.886210 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:34:14.887353 (+ 1143us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:34:14.887360 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:34:14.887365 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 18:34:14.887393 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:34:14.888958 (+ 1565us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:34:14.889456 (+ 498us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:34:14.889460 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:34:14.889462 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:34:14.889558 (+ 96us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:34:14.889816 (+ 258us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:34:14.889820 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:34:14.889824 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:34:14.890047 (+ 223us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:34:14.890176 (+ 129us) server_negotiation.cc:1035] Waiting for connection context
0806 18:34:14.890460 (+ 284us) server_negotiation.cc:299] Negotiation successful
0806 18:34:14.890564 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260806 18:34:14.890713 22254 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:34:14.882788 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.194.62:33617 (local address 127.19.194.1:53013)
0806 18:34:14.883044 (+ 256us) negotiation.cc:107] Waiting for socket to connect
0806 18:34:14.883059 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0806 18:34:14.883172 (+ 113us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:34:14.883438 (+ 266us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:34:14.883441 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:34:14.883646 (+ 205us) client_negotiation.cc:189] Negotiated authn=SASL
0806 18:34:14.883791 (+ 145us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:34:14.883799 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:34:14.884811 (+ 1012us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:34:14.884815 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:34:14.885571 (+ 756us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:34:14.885583 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:34:14.886219 (+ 636us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:34:14.886222 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:34:14.886307 (+ 85us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:34:14.886817 (+ 510us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 18:34:14.886828 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0806 18:34:14.887218 (+ 390us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 18:34:14.889085 (+ 1867us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:34:14.889087 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:34:14.889088 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:34:14.889364 (+ 276us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:34:14.889659 (+ 295us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:34:14.889661 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:34:14.889662 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:34:14.889734 (+ 72us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:34:14.890141 (+ 407us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 18:34:14.890143 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 18:34:14.890233 (+ 90us) client_negotiation.cc:769] Sending connection context
0806 18:34:14.890372 (+ 139us) client_negotiation.cc:240] Negotiation successful
0806 18:34:14.890521 (+ 149us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":168,"thread_start_us":87,"threads_started":1}
WARNING: no policy specified for kudu/127.19.194.2@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.19.194.2@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.19.194.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786041246094603-20232-0/krb5kdc/kudu_127.19.194.2.keytab.
Entry for principal kudu/127.19.194.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786041246094603-20232-0/krb5kdc/kudu_127.19.194.2.keytab.
I20260806 18:34:14.909392 20232 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJW69tn/build/debug/bin/kudu
/tmp/dist-test-taskJW69tn/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786041246094603-20232-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786041246094603-20232-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786041246094603-20232-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786041246094603-20232-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.19.194.2:0
--local_ip_for_outbound_sockets=127.19.194.2
--webserver_interface=127.19.194.2
--webserver_port=0
--tserver_master_addrs=127.19.194.62:33617
--keytab_file=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786041246094603-20232-0/krb5kdc/kudu_127.19.194.2.keytab
--principal=kudu/127.19.194.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.19.194.20:42515
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786041246094603-20232-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786041246094603-20232-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786041246094603-20232-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 18:34:15.017411 22257 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 18:34:15.017691 22257 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 18:34:15.017737 22257 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 18:34:15.021687 22257 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 18:34:15.021739 22257 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 18:34:15.021778 22257 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260806 18:34:15.021840 22257 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260806 18:34:15.021896 22257 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.19.194.2
I20260806 18:34:15.026983 22257 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.194.20:42515
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786041246094603-20232-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786041246094603-20232-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786041246094603-20232-0/krb5kdc/kudu_127.19.194.2.keytab
--principal=kudu/127.19.194.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.19.194.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786041246094603-20232-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.19.194.2
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.19.194.62:33617
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22257
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.19.194.2
--log_dir=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786041246094603-20232-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 17:59:13 UTC on ccycloud.quasar-wvsnzx.root.comops.site
build id 4571028
I20260806 18:34:15.028041 22257 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 18:34:15.029078 22257 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 18:34:15.038448 22263 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:34:15.038450 22265 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 18:34:15.038552 22257 server_base.cc:1034] running on GCE node
W20260806 18:34:15.038482 22262 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 18:34:15.039249 22257 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 18:34:15.039840 22257 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 18:34:15.041002 22257 hybrid_clock.cc:648] HybridClock initialized: now 1786041255040986 us; error 25 us; skew 500 ppm
Aug 06 18:34:15 dist-test-slave-1xrh krb5kdc[21994](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041255, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.194.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 18:34:15.043809 22257 init.cc:377] Logged in from keytab as kudu/127.19.194.2@KRBTEST.COM (short username kudu)
I20260806 18:34:15.044706 22257 webserver.cc:460] Webserver started at http://127.19.194.2:44963/ using document root <none> and password file <none>
I20260806 18:34:15.045243 22257 fs_manager.cc:359] Metadata directory not provided
I20260806 18:34:15.045306 22257 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 18:34:15.045528 22257 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 18:34:15.047490 22257 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786041246094603-20232-0/minicluster-data/ts-1/data/instance:
uuid: "b49bc56953ac498582361f7e613cec38"
format_stamp: "Formatted at 2026-08-06 18:34:15 on dist-test-slave-1xrh"
server_key: "15d9ed8067e03c6b425929af590c3ddb"
server_key_iv: "4ef7bb511febc1546e9867854b8e42d6"
server_key_version: "encryptionkey@0"
I20260806 18:34:15.048122 22257 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786041246094603-20232-0/minicluster-data/ts-1/wal/instance:
uuid: "b49bc56953ac498582361f7e613cec38"
format_stamp: "Formatted at 2026-08-06 18:34:15 on dist-test-slave-1xrh"
server_key: "15d9ed8067e03c6b425929af590c3ddb"
server_key_iv: "4ef7bb511febc1546e9867854b8e42d6"
server_key_version: "encryptionkey@0"
I20260806 18:34:15.051589 22257 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.000s
I20260806 18:34:15.053715 22272 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 18:34:15.054826 22257 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260806 18:34:15.054955 22257 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786041246094603-20232-0/minicluster-data/ts-1/data,/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786041246094603-20232-0/minicluster-data/ts-1/wal
uuid: "b49bc56953ac498582361f7e613cec38"
format_stamp: "Formatted at 2026-08-06 18:34:15 on dist-test-slave-1xrh"
server_key: "15d9ed8067e03c6b425929af590c3ddb"
server_key_iv: "4ef7bb511febc1546e9867854b8e42d6"
server_key_version: "encryptionkey@0"
I20260806 18:34:15.055060 22257 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786041246094603-20232-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786041246094603-20232-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786041246094603-20232-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 18:34:15.063977 22257 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 18:34:15.066972 22257 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 18:34:15.067229 22257 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 18:34:15.068619 22257 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 18:34:15.068688 22257 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 18:34:15.068754 22257 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 18:34:15.068802 22257 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 06 18:34:15 dist-test-slave-1xrh krb5kdc[21994](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041255, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.194.2@KRBTEST.COM for kudu/127.19.194.62@KRBTEST.COM
I20260806 18:34:15.079468 22257 rpc_server.cc:307] RPC server started. Bound to: 127.19.194.2:38833
I20260806 18:34:15.079612 22391 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.194.2:38833 every 8 connection(s)
I20260806 18:34:15.080569 22257 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786041246094603-20232-0/minicluster-data/ts-1/data/info.pb
I20260806 18:34:15.082516 22090 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:34:15.069295 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.194.2:46929 (local address 127.19.194.62:33617)
0806 18:34:15.069461 (+ 166us) server_negotiation.cc:206] Beginning negotiation
0806 18:34:15.069468 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 18:34:15.070605 (+ 1137us) server_negotiation.cc:407] Connection header received
0806 18:34:15.071810 (+ 1205us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:34:15.071815 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:34:15.071887 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:34:15.071995 (+ 108us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:34:15.073205 (+ 1210us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:34:15.073759 (+ 554us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:34:15.075050 (+ 1291us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:34:15.075420 (+ 370us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:34:15.075549 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:34:15.078532 (+ 2983us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:34:15.078538 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:34:15.078541 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 18:34:15.078574 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:34:15.080429 (+ 1855us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:34:15.081104 (+ 675us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:34:15.081110 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:34:15.081111 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:34:15.081192 (+ 81us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:34:15.081599 (+ 407us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:34:15.081602 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:34:15.081604 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:34:15.081783 (+ 179us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:34:15.081869 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0806 18:34:15.082302 (+ 433us) server_negotiation.cc:299] Negotiation successful
0806 18:34:15.082399 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260806 18:34:15.083336 22281 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:34:15.070048 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.194.62:33617 (local address 127.19.194.2:46929)
0806 18:34:15.070465 (+ 417us) negotiation.cc:107] Waiting for socket to connect
0806 18:34:15.070513 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0806 18:34:15.071583 (+ 1070us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:34:15.072186 (+ 603us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:34:15.072192 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:34:15.072492 (+ 300us) client_negotiation.cc:189] Negotiated authn=SASL
0806 18:34:15.073059 (+ 567us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:34:15.073072 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:34:15.073889 (+ 817us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:34:15.073893 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:34:15.074910 (+ 1017us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:34:15.074920 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:34:15.075598 (+ 678us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:34:15.075603 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:34:15.075755 (+ 152us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:34:15.076313 (+ 558us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 18:34:15.076328 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0806 18:34:15.078341 (+ 2013us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 18:34:15.080542 (+ 2201us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:34:15.080547 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:34:15.080557 (+ 10us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:34:15.080950 (+ 393us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:34:15.081280 (+ 330us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:34:15.081284 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:34:15.081286 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:34:15.081484 (+ 198us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:34:15.081887 (+ 403us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 18:34:15.081892 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 18:34:15.082085 (+ 193us) client_negotiation.cc:769] Sending connection context
0806 18:34:15.082252 (+ 167us) client_negotiation.cc:240] Negotiation successful
0806 18:34:15.082477 (+ 225us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":228,"thread_start_us":118,"threads_started":1}
I20260806 18:34:15.084564 22392 heartbeater.cc:347] Connected to a master server at 127.19.194.62:33617
I20260806 18:34:15.084721 22392 heartbeater.cc:464] Registering TS with master...
I20260806 18:34:15.084920 20232 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJW69tn/build/debug/bin/kudu as pid 22257
I20260806 18:34:15.085034 20232 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786041246094603-20232-0/minicluster-data/ts-1/wal/instance
I20260806 18:34:15.085287 22392 heartbeater.cc:511] Master 127.19.194.62:33617 requested a full tablet report, sending...
I20260806 18:34:15.085377 20232 external_mini_cluster.cc:1428] Setting key 3ff3c7aa4dca164168730385732617f1
I20260806 18:34:15.086364 22038 ts_manager.cc:194] Registered new tserver with Master: b49bc56953ac498582361f7e613cec38 (127.19.194.2:38833)
I20260806 18:34:15.087080 22038 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.19.194.2@KRBTEST.COM'} at 127.19.194.2:46929
I20260806 18:34:15.094916 22398 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:34:15.086895 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.194.62:33617 (local address 127.19.194.2:34951)
0806 18:34:15.087182 (+ 287us) negotiation.cc:107] Waiting for socket to connect
0806 18:34:15.087200 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0806 18:34:15.087305 (+ 105us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:34:15.087719 (+ 414us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:34:15.087722 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:34:15.087938 (+ 216us) client_negotiation.cc:189] Negotiated authn=SASL
0806 18:34:15.088107 (+ 169us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:34:15.088129 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:34:15.089128 (+ 999us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:34:15.089131 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:34:15.089851 (+ 720us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:34:15.089860 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:34:15.090496 (+ 636us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:34:15.090498 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:34:15.090564 (+ 66us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:34:15.091097 (+ 533us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 18:34:15.091114 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0806 18:34:15.091446 (+ 332us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 18:34:15.093433 (+ 1987us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:34:15.093436 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:34:15.093437 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:34:15.093630 (+ 193us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:34:15.093932 (+ 302us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:34:15.093934 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:34:15.093935 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:34:15.094014 (+ 79us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:34:15.094526 (+ 512us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 18:34:15.094528 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 18:34:15.094600 (+ 72us) client_negotiation.cc:769] Sending connection context
0806 18:34:15.094677 (+ 77us) client_negotiation.cc:240] Negotiation successful
0806 18:34:15.094778 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":187,"mutex_wait_us":9,"thread_start_us":104,"threads_started":1}
I20260806 18:34:15.095068 22090 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:34:15.086870 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.194.2:34951 (local address 127.19.194.62:33617)
0806 18:34:15.086984 (+ 114us) server_negotiation.cc:206] Beginning negotiation
0806 18:34:15.086991 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 18:34:15.087269 (+ 278us) server_negotiation.cc:407] Connection header received
0806 18:34:15.087456 (+ 187us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:34:15.087459 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:34:15.087518 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:34:15.087559 (+ 41us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:34:15.088252 (+ 693us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:34:15.088958 (+ 706us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:34:15.089961 (+ 1003us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:34:15.090375 (+ 414us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:34:15.090502 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:34:15.091599 (+ 1097us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:34:15.091605 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:34:15.091611 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 18:34:15.091659 (+ 48us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:34:15.093323 (+ 1664us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:34:15.093727 (+ 404us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:34:15.093732 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:34:15.093734 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:34:15.093828 (+ 94us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:34:15.094102 (+ 274us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:34:15.094106 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:34:15.094108 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:34:15.094427 (+ 319us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:34:15.094542 (+ 115us) server_negotiation.cc:1035] Waiting for connection context
0806 18:34:15.094766 (+ 224us) server_negotiation.cc:299] Negotiation successful
0806 18:34:15.094887 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
WARNING: no policy specified for kudu/127.19.194.3@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.19.194.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.19.194.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786041246094603-20232-0/krb5kdc/kudu_127.19.194.3.keytab.
Entry for principal kudu/127.19.194.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786041246094603-20232-0/krb5kdc/kudu_127.19.194.3.keytab.
I20260806 18:34:15.114051 20232 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJW69tn/build/debug/bin/kudu
/tmp/dist-test-taskJW69tn/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786041246094603-20232-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786041246094603-20232-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786041246094603-20232-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786041246094603-20232-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.19.194.3:0
--local_ip_for_outbound_sockets=127.19.194.3
--webserver_interface=127.19.194.3
--webserver_port=0
--tserver_master_addrs=127.19.194.62:33617
--keytab_file=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786041246094603-20232-0/krb5kdc/kudu_127.19.194.3.keytab
--principal=kudu/127.19.194.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.19.194.20:42515
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786041246094603-20232-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786041246094603-20232-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786041246094603-20232-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 18:34:15.222985 22401 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 18:34:15.223266 22401 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 18:34:15.223315 22401 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 18:34:15.227200 22401 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 18:34:15.227245 22401 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 18:34:15.227281 22401 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260806 18:34:15.227295 22401 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260806 18:34:15.227330 22401 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.19.194.3
I20260806 18:34:15.232427 22401 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.194.20:42515
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786041246094603-20232-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786041246094603-20232-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786041246094603-20232-0/krb5kdc/kudu_127.19.194.3.keytab
--principal=kudu/127.19.194.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.19.194.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786041246094603-20232-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.19.194.3
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.19.194.62:33617
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22401
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.19.194.3
--log_dir=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786041246094603-20232-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 17:59:13 UTC on ccycloud.quasar-wvsnzx.root.comops.site
build id 4571028
I20260806 18:34:15.233289 22401 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 18:34:15.234192 22401 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 18:34:15.243388 22406 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:34:15.243397 22409 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 18:34:15.243438 22401 server_base.cc:1034] running on GCE node
W20260806 18:34:15.243405 22407 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 18:34:15.243978 22401 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 18:34:15.244607 22401 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 18:34:15.245786 22401 hybrid_clock.cc:648] HybridClock initialized: now 1786041255245773 us; error 29 us; skew 500 ppm
Aug 06 18:34:15 dist-test-slave-1xrh krb5kdc[21994](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041255, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.194.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 18:34:15.248540 22401 init.cc:377] Logged in from keytab as kudu/127.19.194.3@KRBTEST.COM (short username kudu)
I20260806 18:34:15.249378 22401 webserver.cc:460] Webserver started at http://127.19.194.3:42509/ using document root <none> and password file <none>
I20260806 18:34:15.249872 22401 fs_manager.cc:359] Metadata directory not provided
I20260806 18:34:15.249926 22401 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 18:34:15.250084 22401 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 18:34:15.251955 22401 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786041246094603-20232-0/minicluster-data/ts-2/data/instance:
uuid: "c5fb6bfd8ec4401fb0f9b3cf91ae5ef3"
format_stamp: "Formatted at 2026-08-06 18:34:15 on dist-test-slave-1xrh"
server_key: "223cab3d3455f83f6730136964df2762"
server_key_iv: "511dbd7c8188d17b3506c57b14cb734c"
server_key_version: "encryptionkey@0"
I20260806 18:34:15.252444 22401 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786041246094603-20232-0/minicluster-data/ts-2/wal/instance:
uuid: "c5fb6bfd8ec4401fb0f9b3cf91ae5ef3"
format_stamp: "Formatted at 2026-08-06 18:34:15 on dist-test-slave-1xrh"
server_key: "223cab3d3455f83f6730136964df2762"
server_key_iv: "511dbd7c8188d17b3506c57b14cb734c"
server_key_version: "encryptionkey@0"
I20260806 18:34:15.255980 22401 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.001s
I20260806 18:34:15.258224 22416 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 18:34:15.259474 22401 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260806 18:34:15.259565 22401 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786041246094603-20232-0/minicluster-data/ts-2/data,/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786041246094603-20232-0/minicluster-data/ts-2/wal
uuid: "c5fb6bfd8ec4401fb0f9b3cf91ae5ef3"
format_stamp: "Formatted at 2026-08-06 18:34:15 on dist-test-slave-1xrh"
server_key: "223cab3d3455f83f6730136964df2762"
server_key_iv: "511dbd7c8188d17b3506c57b14cb734c"
server_key_version: "encryptionkey@0"
I20260806 18:34:15.259644 22401 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786041246094603-20232-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786041246094603-20232-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786041246094603-20232-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 18:34:15.285727 22401 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 18:34:15.288470 22401 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 18:34:15.288640 22401 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 18:34:15.290160 22401 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 18:34:15.290242 22401 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 18:34:15.290310 22401 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 18:34:15.290359 22401 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 06 18:34:15 dist-test-slave-1xrh krb5kdc[21994](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041255, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.194.3@KRBTEST.COM for kudu/127.19.194.62@KRBTEST.COM
I20260806 18:34:15.301043 22401 rpc_server.cc:307] RPC server started. Bound to: 127.19.194.3:43573
I20260806 18:34:15.301714 22533 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.194.3:43573 every 8 connection(s)
I20260806 18:34:15.301954 22401 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786041246094603-20232-0/minicluster-data/ts-2/data/info.pb
I20260806 18:34:15.303700 22090 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:34:15.290821 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.194.3:56707 (local address 127.19.194.62:33617)
0806 18:34:15.290988 (+ 167us) server_negotiation.cc:206] Beginning negotiation
0806 18:34:15.290994 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 18:34:15.291915 (+ 921us) server_negotiation.cc:407] Connection header received
0806 18:34:15.292866 (+ 951us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:34:15.292869 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:34:15.292933 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:34:15.293038 (+ 105us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:34:15.294787 (+ 1749us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:34:15.295682 (+ 895us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:34:15.296751 (+ 1069us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:34:15.297046 (+ 295us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:34:15.297163 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:34:15.300033 (+ 2870us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:34:15.300038 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:34:15.300040 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 18:34:15.300066 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:34:15.301542 (+ 1476us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:34:15.302120 (+ 578us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:34:15.302124 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:34:15.302125 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:34:15.302228 (+ 103us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:34:15.302628 (+ 400us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:34:15.302633 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:34:15.302635 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:34:15.302853 (+ 218us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:34:15.302959 (+ 106us) server_negotiation.cc:1035] Waiting for connection context
0806 18:34:15.303488 (+ 529us) server_negotiation.cc:299] Negotiation successful
0806 18:34:15.303586 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260806 18:34:15.304440 22425 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:34:15.291276 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.194.62:33617 (local address 127.19.194.3:56707)
0806 18:34:15.291745 (+ 469us) negotiation.cc:107] Waiting for socket to connect
0806 18:34:15.291794 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0806 18:34:15.292627 (+ 833us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:34:15.293370 (+ 743us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:34:15.293380 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:34:15.293893 (+ 513us) client_negotiation.cc:189] Negotiated authn=SASL
0806 18:34:15.294591 (+ 698us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:34:15.294615 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:34:15.295810 (+ 1195us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:34:15.295813 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:34:15.296627 (+ 814us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:34:15.296637 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:34:15.297212 (+ 575us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:34:15.297216 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:34:15.297362 (+ 146us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:34:15.297901 (+ 539us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 18:34:15.297934 (+ 33us) client_negotiation.cc:656] calling sasl_client_start()
0806 18:34:15.299846 (+ 1912us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 18:34:15.301650 (+ 1804us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:34:15.301655 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:34:15.301668 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:34:15.301991 (+ 323us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:34:15.302334 (+ 343us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:34:15.302339 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:34:15.302343 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:34:15.302521 (+ 178us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:34:15.302950 (+ 429us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 18:34:15.302955 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 18:34:15.303282 (+ 327us) client_negotiation.cc:769] Sending connection context
0806 18:34:15.303440 (+ 158us) client_negotiation.cc:240] Negotiation successful
0806 18:34:15.303670 (+ 230us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":281,"thread_start_us":122,"threads_started":1}
I20260806 18:34:15.305608 22534 heartbeater.cc:347] Connected to a master server at 127.19.194.62:33617
I20260806 18:34:15.305764 22534 heartbeater.cc:464] Registering TS with master...
I20260806 18:34:15.306193 22534 heartbeater.cc:511] Master 127.19.194.62:33617 requested a full tablet report, sending...
I20260806 18:34:15.307226 22040 ts_manager.cc:194] Registered new tserver with Master: c5fb6bfd8ec4401fb0f9b3cf91ae5ef3 (127.19.194.3:43573)
I20260806 18:34:15.307948 22040 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.19.194.3@KRBTEST.COM'} at 127.19.194.3:56707
I20260806 18:34:15.310119 20232 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJW69tn/build/debug/bin/kudu as pid 22401
I20260806 18:34:15.310245 20232 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786041246094603-20232-0/minicluster-data/ts-2/wal/instance
I20260806 18:34:15.310598 20232 external_mini_cluster.cc:1428] Setting key 081681171e7fd2154d1a39434ef50d48
I20260806 18:34:15.315485 20232 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260806 18:34:15.316044 22090 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:34:15.307740 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.194.3:49817 (local address 127.19.194.62:33617)
0806 18:34:15.307851 (+ 111us) server_negotiation.cc:206] Beginning negotiation
0806 18:34:15.307855 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 18:34:15.308090 (+ 235us) server_negotiation.cc:407] Connection header received
0806 18:34:15.308186 (+ 96us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:34:15.308188 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:34:15.308225 (+ 37us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:34:15.308329 (+ 104us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:34:15.308854 (+ 525us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:34:15.309304 (+ 450us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:34:15.310484 (+ 1180us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:34:15.310877 (+ 393us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:34:15.310983 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:34:15.312369 (+ 1386us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:34:15.312373 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:34:15.312375 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 18:34:15.312404 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:34:15.314164 (+ 1760us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:34:15.314689 (+ 525us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:34:15.314706 (+ 17us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:34:15.314708 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:34:15.314794 (+ 86us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:34:15.315080 (+ 286us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:34:15.315097 (+ 17us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:34:15.315098 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:34:15.315354 (+ 256us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:34:15.315446 (+ 92us) server_negotiation.cc:1035] Waiting for connection context
0806 18:34:15.315782 (+ 336us) server_negotiation.cc:299] Negotiation successful
0806 18:34:15.315884 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260806 18:34:15.316609 22540 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:34:15.307648 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.194.62:33617 (local address 127.19.194.3:49817)
0806 18:34:15.308005 (+ 357us) negotiation.cc:107] Waiting for socket to connect
0806 18:34:15.308020 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0806 18:34:15.308110 (+ 90us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:34:15.308348 (+ 238us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:34:15.308351 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:34:15.308569 (+ 218us) client_negotiation.cc:189] Negotiated authn=SASL
0806 18:34:15.308740 (+ 171us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:34:15.308748 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:34:15.309432 (+ 684us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:34:15.309435 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:34:15.310349 (+ 914us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:34:15.310360 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:34:15.311001 (+ 641us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:34:15.311008 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:34:15.311159 (+ 151us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:34:15.311697 (+ 538us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 18:34:15.311715 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0806 18:34:15.312200 (+ 485us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 18:34:15.314285 (+ 2085us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:34:15.314289 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:34:15.314292 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:34:15.314591 (+ 299us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:34:15.314879 (+ 288us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:34:15.314882 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:34:15.314899 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:34:15.314990 (+ 91us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:34:15.315508 (+ 518us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 18:34:15.315512 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 18:34:15.315602 (+ 90us) client_negotiation.cc:769] Sending connection context
0806 18:34:15.316164 (+ 562us) client_negotiation.cc:240] Negotiation successful
0806 18:34:15.316316 (+ 152us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":258,"mutex_wait_us":24,"thread_start_us":85,"threads_started":1}
Aug 06 18:34:15 dist-test-slave-1xrh krb5kdc[21994](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041255, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 06 18:34:15 dist-test-slave-1xrh krb5kdc[21994](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041255, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.19.194.62@KRBTEST.COM
I20260806 18:34:15.344460 22090 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:34:15.336339 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40686 (local address 127.19.194.62:33617)
0806 18:34:15.336471 (+ 132us) server_negotiation.cc:206] Beginning negotiation
0806 18:34:15.336475 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 18:34:15.336487 (+ 12us) server_negotiation.cc:407] Connection header received
0806 18:34:15.336525 (+ 38us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:34:15.336527 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:34:15.336563 (+ 36us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:34:15.336631 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:34:15.337560 (+ 929us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:34:15.338057 (+ 497us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:34:15.338901 (+ 844us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:34:15.339116 (+ 215us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:34:15.339260 (+ 144us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:34:15.341500 (+ 2240us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:34:15.341504 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:34:15.341506 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 18:34:15.341534 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:34:15.342758 (+ 1224us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:34:15.343231 (+ 473us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:34:15.343235 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:34:15.343237 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:34:15.343317 (+ 80us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:34:15.343592 (+ 275us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:34:15.343596 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:34:15.343597 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:34:15.343773 (+ 176us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:34:15.343982 (+ 209us) server_negotiation.cc:1035] Waiting for connection context
0806 18:34:15.344179 (+ 197us) server_negotiation.cc:299] Negotiation successful
0806 18:34:15.344297 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260806 18:34:15.347285 22040 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:40686:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260806 18:34:15.350323 22040 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 18:34:15.364862 22553 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:34:15.358536 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.194.3:43573 (local address 127.0.0.1:36884)
0806 18:34:15.358957 (+ 421us) negotiation.cc:107] Waiting for socket to connect
0806 18:34:15.359010 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0806 18:34:15.359164 (+ 154us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:34:15.360200 (+ 1036us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:34:15.360206 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:34:15.360229 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 18:34:15.360379 (+ 150us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:34:15.360386 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:34:15.361940 (+ 1554us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:34:15.361943 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:34:15.363277 (+ 1334us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:34:15.363288 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:34:15.364374 (+ 1086us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:34:15.364377 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:34:15.364476 (+ 99us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:34:15.364509 (+ 33us) client_negotiation.cc:769] Sending connection context
0806 18:34:15.364562 (+ 53us) client_negotiation.cc:240] Negotiation successful
0806 18:34:15.364717 (+ 155us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":245,"mutex_wait_us":90,"thread_start_us":98,"threads_started":1}
I20260806 18:34:15.364861 22556 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:34:15.359356 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55086 (local address 127.19.194.2:38833)
0806 18:34:15.359844 (+ 488us) server_negotiation.cc:206] Beginning negotiation
0806 18:34:15.359858 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0806 18:34:15.359881 (+ 23us) server_negotiation.cc:407] Connection header received
0806 18:34:15.359978 (+ 97us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:34:15.359984 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:34:15.360164 (+ 180us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:34:15.360587 (+ 423us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 18:34:15.360725 (+ 138us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:34:15.361517 (+ 792us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:34:15.363296 (+ 1779us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:34:15.364160 (+ 864us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:34:15.364290 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:34:15.364479 (+ 189us) server_negotiation.cc:1035] Waiting for connection context
0806 18:34:15.364601 (+ 122us) server_negotiation.cc:299] Negotiation successful
0806 18:34:15.364699 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":391,"thread_start_us":94,"threads_started":1}
I20260806 18:34:15.365130 22554 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:34:15.358684 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36884 (local address 127.19.194.3:43573)
0806 18:34:15.359679 (+ 995us) server_negotiation.cc:206] Beginning negotiation
0806 18:34:15.359692 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0806 18:34:15.359716 (+ 24us) server_negotiation.cc:407] Connection header received
0806 18:34:15.359794 (+ 78us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:34:15.359801 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:34:15.360021 (+ 220us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:34:15.360538 (+ 517us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 18:34:15.360681 (+ 143us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:34:15.361823 (+ 1142us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:34:15.363447 (+ 1624us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:34:15.364241 (+ 794us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:34:15.364689 (+ 448us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:34:15.364830 (+ 141us) server_negotiation.cc:1035] Waiting for connection context
0806 18:34:15.364880 (+ 50us) server_negotiation.cc:299] Negotiation successful
0806 18:34:15.364987 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":885,"thread_start_us":117,"threads_started":1}
I20260806 18:34:15.364902 22552 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:34:15.359273 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.194.1:37593 (local address 127.0.0.1:49642)
0806 18:34:15.360101 (+ 828us) negotiation.cc:107] Waiting for socket to connect
0806 18:34:15.360116 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0806 18:34:15.360197 (+ 81us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:34:15.360718 (+ 521us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:34:15.360721 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:34:15.360738 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 18:34:15.360824 (+ 86us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:34:15.360829 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:34:15.362119 (+ 1290us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:34:15.362123 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:34:15.363434 (+ 1311us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:34:15.363442 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:34:15.364558 (+ 1116us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:34:15.364564 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:34:15.364655 (+ 91us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:34:15.364675 (+ 20us) client_negotiation.cc:769] Sending connection context
0806 18:34:15.364740 (+ 65us) client_negotiation.cc:240] Negotiation successful
0806 18:34:15.364795 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":743,"mutex_wait_us":21,"thread_start_us":93,"threads_started":1}
I20260806 18:34:15.365631 22557 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:34:15.359425 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49642 (local address 127.19.194.1:37593)
0806 18:34:15.359845 (+ 420us) server_negotiation.cc:206] Beginning negotiation
0806 18:34:15.359862 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0806 18:34:15.360295 (+ 433us) server_negotiation.cc:407] Connection header received
0806 18:34:15.360382 (+ 87us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:34:15.360387 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:34:15.360573 (+ 186us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:34:15.360732 (+ 159us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 18:34:15.360923 (+ 191us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:34:15.361995 (+ 1072us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:34:15.363548 (+ 1553us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:34:15.364203 (+ 655us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:34:15.364327 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:34:15.364491 (+ 164us) server_negotiation.cc:1035] Waiting for connection context
0806 18:34:15.365353 (+ 862us) server_negotiation.cc:299] Negotiation successful
0806 18:34:15.365454 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":318,"thread_start_us":103,"threads_started":1}
I20260806 18:34:15.365653 22555 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:34:15.358546 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.194.2:38833 (local address 127.0.0.1:55086)
0806 18:34:15.359508 (+ 962us) negotiation.cc:107] Waiting for socket to connect
0806 18:34:15.359523 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0806 18:34:15.359612 (+ 89us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:34:15.360326 (+ 714us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:34:15.360329 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:34:15.360349 (+ 20us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 18:34:15.360451 (+ 102us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:34:15.360457 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:34:15.361626 (+ 1169us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:34:15.361630 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:34:15.363125 (+ 1495us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:34:15.363170 (+ 45us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:34:15.364304 (+ 1134us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:34:15.364308 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:34:15.364413 (+ 105us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:34:15.364450 (+ 37us) client_negotiation.cc:769] Sending connection context
0806 18:34:15.364574 (+ 124us) client_negotiation.cc:240] Negotiation successful
0806 18:34:15.365242 (+ 668us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":899,"mutex_wait_us":61,"thread_start_us":69,"threads_started":1}
I20260806 18:34:15.367933 22468 tablet_service.cc:1467] Processing CreateTablet for tablet 51387960396d495ab763cdeee6fe1006 (DEFAULT_TABLE table=test-table [id=1aff44fa39374b29a01198a905d82f93]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 18:34:15.368454 22324 tablet_service.cc:1467] Processing CreateTablet for tablet 51387960396d495ab763cdeee6fe1006 (DEFAULT_TABLE table=test-table [id=1aff44fa39374b29a01198a905d82f93]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 18:34:15.368454 22180 tablet_service.cc:1467] Processing CreateTablet for tablet 51387960396d495ab763cdeee6fe1006 (DEFAULT_TABLE table=test-table [id=1aff44fa39374b29a01198a905d82f93]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 18:34:15.368913 22468 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 51387960396d495ab763cdeee6fe1006. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 18:34:15.369447 22180 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 51387960396d495ab763cdeee6fe1006. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 18:34:15.369447 22324 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 51387960396d495ab763cdeee6fe1006. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 18:34:15.375664 22558 tablet_bootstrap.cc:492] T 51387960396d495ab763cdeee6fe1006 P c5fb6bfd8ec4401fb0f9b3cf91ae5ef3: Bootstrap starting.
I20260806 18:34:15.377055 22560 tablet_bootstrap.cc:492] T 51387960396d495ab763cdeee6fe1006 P 39d31d4b04c14099bf89458586ff793c: Bootstrap starting.
I20260806 18:34:15.377817 22559 tablet_bootstrap.cc:492] T 51387960396d495ab763cdeee6fe1006 P b49bc56953ac498582361f7e613cec38: Bootstrap starting.
I20260806 18:34:15.378348 22558 tablet_bootstrap.cc:654] T 51387960396d495ab763cdeee6fe1006 P c5fb6bfd8ec4401fb0f9b3cf91ae5ef3: Neither blocks nor log segments found. Creating new log.
I20260806 18:34:15.379243 22560 tablet_bootstrap.cc:654] T 51387960396d495ab763cdeee6fe1006 P 39d31d4b04c14099bf89458586ff793c: Neither blocks nor log segments found. Creating new log.
I20260806 18:34:15.379249 22558 log.cc:824] T 51387960396d495ab763cdeee6fe1006 P c5fb6bfd8ec4401fb0f9b3cf91ae5ef3: Log is configured to *not* fsync() on all Append() calls
I20260806 18:34:15.379601 22559 tablet_bootstrap.cc:654] T 51387960396d495ab763cdeee6fe1006 P b49bc56953ac498582361f7e613cec38: Neither blocks nor log segments found. Creating new log.
I20260806 18:34:15.380036 22560 log.cc:824] T 51387960396d495ab763cdeee6fe1006 P 39d31d4b04c14099bf89458586ff793c: Log is configured to *not* fsync() on all Append() calls
I20260806 18:34:15.380479 22559 log.cc:824] T 51387960396d495ab763cdeee6fe1006 P b49bc56953ac498582361f7e613cec38: Log is configured to *not* fsync() on all Append() calls
I20260806 18:34:15.381815 22558 tablet_bootstrap.cc:492] T 51387960396d495ab763cdeee6fe1006 P c5fb6bfd8ec4401fb0f9b3cf91ae5ef3: No bootstrap required, opened a new log
I20260806 18:34:15.382006 22558 ts_tablet_manager.cc:1397] T 51387960396d495ab763cdeee6fe1006 P c5fb6bfd8ec4401fb0f9b3cf91ae5ef3: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260806 18:34:15.382507 22560 tablet_bootstrap.cc:492] T 51387960396d495ab763cdeee6fe1006 P 39d31d4b04c14099bf89458586ff793c: No bootstrap required, opened a new log
I20260806 18:34:15.382507 22559 tablet_bootstrap.cc:492] T 51387960396d495ab763cdeee6fe1006 P b49bc56953ac498582361f7e613cec38: No bootstrap required, opened a new log
I20260806 18:34:15.382699 22559 ts_tablet_manager.cc:1397] T 51387960396d495ab763cdeee6fe1006 P b49bc56953ac498582361f7e613cec38: Time spent bootstrapping tablet: real 0.005s user 0.003s sys 0.000s
I20260806 18:34:15.382712 22560 ts_tablet_manager.cc:1397] T 51387960396d495ab763cdeee6fe1006 P 39d31d4b04c14099bf89458586ff793c: Time spent bootstrapping tablet: real 0.006s user 0.001s sys 0.003s
I20260806 18:34:15.385713 22560 raft_consensus.cc:348] T 51387960396d495ab763cdeee6fe1006 P 39d31d4b04c14099bf89458586ff793c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c5fb6bfd8ec4401fb0f9b3cf91ae5ef3" member_type: VOTER last_known_addr { host: "127.19.194.3" port: 43573 } } peers { permanent_uuid: "b49bc56953ac498582361f7e613cec38" member_type: VOTER last_known_addr { host: "127.19.194.2" port: 38833 } } peers { permanent_uuid: "39d31d4b04c14099bf89458586ff793c" member_type: VOTER last_known_addr { host: "127.19.194.1" port: 37593 } }
I20260806 18:34:15.385970 22560 raft_consensus.cc:374] T 51387960396d495ab763cdeee6fe1006 P 39d31d4b04c14099bf89458586ff793c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 18:34:15.386034 22560 raft_consensus.cc:729] T 51387960396d495ab763cdeee6fe1006 P 39d31d4b04c14099bf89458586ff793c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 39d31d4b04c14099bf89458586ff793c, State: Initialized, Role: FOLLOWER
I20260806 18:34:15.386006 22558 raft_consensus.cc:348] T 51387960396d495ab763cdeee6fe1006 P c5fb6bfd8ec4401fb0f9b3cf91ae5ef3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c5fb6bfd8ec4401fb0f9b3cf91ae5ef3" member_type: VOTER last_known_addr { host: "127.19.194.3" port: 43573 } } peers { permanent_uuid: "b49bc56953ac498582361f7e613cec38" member_type: VOTER last_known_addr { host: "127.19.194.2" port: 38833 } } peers { permanent_uuid: "39d31d4b04c14099bf89458586ff793c" member_type: VOTER last_known_addr { host: "127.19.194.1" port: 37593 } }
I20260806 18:34:15.386255 22558 raft_consensus.cc:374] T 51387960396d495ab763cdeee6fe1006 P c5fb6bfd8ec4401fb0f9b3cf91ae5ef3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 18:34:15.386312 22558 raft_consensus.cc:729] T 51387960396d495ab763cdeee6fe1006 P c5fb6bfd8ec4401fb0f9b3cf91ae5ef3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c5fb6bfd8ec4401fb0f9b3cf91ae5ef3, State: Initialized, Role: FOLLOWER
I20260806 18:34:15.386535 22560 consensus_queue.cc:260] T 51387960396d495ab763cdeee6fe1006 P 39d31d4b04c14099bf89458586ff793c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c5fb6bfd8ec4401fb0f9b3cf91ae5ef3" member_type: VOTER last_known_addr { host: "127.19.194.3" port: 43573 } } peers { permanent_uuid: "b49bc56953ac498582361f7e613cec38" member_type: VOTER last_known_addr { host: "127.19.194.2" port: 38833 } } peers { permanent_uuid: "39d31d4b04c14099bf89458586ff793c" member_type: VOTER last_known_addr { host: "127.19.194.1" port: 37593 } }
I20260806 18:34:15.386488 22559 raft_consensus.cc:348] T 51387960396d495ab763cdeee6fe1006 P b49bc56953ac498582361f7e613cec38 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c5fb6bfd8ec4401fb0f9b3cf91ae5ef3" member_type: VOTER last_known_addr { host: "127.19.194.3" port: 43573 } } peers { permanent_uuid: "b49bc56953ac498582361f7e613cec38" member_type: VOTER last_known_addr { host: "127.19.194.2" port: 38833 } } peers { permanent_uuid: "39d31d4b04c14099bf89458586ff793c" member_type: VOTER last_known_addr { host: "127.19.194.1" port: 37593 } }
I20260806 18:34:15.386698 22559 raft_consensus.cc:374] T 51387960396d495ab763cdeee6fe1006 P b49bc56953ac498582361f7e613cec38 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 18:34:15.386755 22559 raft_consensus.cc:729] T 51387960396d495ab763cdeee6fe1006 P b49bc56953ac498582361f7e613cec38 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b49bc56953ac498582361f7e613cec38, State: Initialized, Role: FOLLOWER
I20260806 18:34:15.386711 22558 consensus_queue.cc:260] T 51387960396d495ab763cdeee6fe1006 P c5fb6bfd8ec4401fb0f9b3cf91ae5ef3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c5fb6bfd8ec4401fb0f9b3cf91ae5ef3" member_type: VOTER last_known_addr { host: "127.19.194.3" port: 43573 } } peers { permanent_uuid: "b49bc56953ac498582361f7e613cec38" member_type: VOTER last_known_addr { host: "127.19.194.2" port: 38833 } } peers { permanent_uuid: "39d31d4b04c14099bf89458586ff793c" member_type: VOTER last_known_addr { host: "127.19.194.1" port: 37593 } }
I20260806 18:34:15.387212 22559 consensus_queue.cc:260] T 51387960396d495ab763cdeee6fe1006 P b49bc56953ac498582361f7e613cec38 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c5fb6bfd8ec4401fb0f9b3cf91ae5ef3" member_type: VOTER last_known_addr { host: "127.19.194.3" port: 43573 } } peers { permanent_uuid: "b49bc56953ac498582361f7e613cec38" member_type: VOTER last_known_addr { host: "127.19.194.2" port: 38833 } } peers { permanent_uuid: "39d31d4b04c14099bf89458586ff793c" member_type: VOTER last_known_addr { host: "127.19.194.1" port: 37593 } }
I20260806 18:34:15.387691 22558 ts_tablet_manager.cc:1428] T 51387960396d495ab763cdeee6fe1006 P c5fb6bfd8ec4401fb0f9b3cf91ae5ef3: Time spent starting tablet: real 0.006s user 0.001s sys 0.005s
I20260806 18:34:15.387847 22560 ts_tablet_manager.cc:1428] T 51387960396d495ab763cdeee6fe1006 P 39d31d4b04c14099bf89458586ff793c: Time spent starting tablet: real 0.005s user 0.004s sys 0.001s
I20260806 18:34:15.388028 22534 heartbeater.cc:503] Master 127.19.194.62:33617 was elected leader, sending a full tablet report...
I20260806 18:34:15.388056 22559 ts_tablet_manager.cc:1428] T 51387960396d495ab763cdeee6fe1006 P b49bc56953ac498582361f7e613cec38: Time spent starting tablet: real 0.005s user 0.006s sys 0.000s
I20260806 18:34:15.388185 22248 heartbeater.cc:503] Master 127.19.194.62:33617 was elected leader, sending a full tablet report...
I20260806 18:34:15.388370 22392 heartbeater.cc:503] Master 127.19.194.62:33617 was elected leader, sending a full tablet report...
I20260806 18:34:15.395339 22566 raft_consensus.cc:482] T 51387960396d495ab763cdeee6fe1006 P b49bc56953ac498582361f7e613cec38 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 18:34:15.395481 22566 raft_consensus.cc:504] T 51387960396d495ab763cdeee6fe1006 P b49bc56953ac498582361f7e613cec38 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c5fb6bfd8ec4401fb0f9b3cf91ae5ef3" member_type: VOTER last_known_addr { host: "127.19.194.3" port: 43573 } } peers { permanent_uuid: "b49bc56953ac498582361f7e613cec38" member_type: VOTER last_known_addr { host: "127.19.194.2" port: 38833 } } peers { permanent_uuid: "39d31d4b04c14099bf89458586ff793c" member_type: VOTER last_known_addr { host: "127.19.194.1" port: 37593 } }
I20260806 18:34:15.396342 22566 leader_election.cc:288] T 51387960396d495ab763cdeee6fe1006 P b49bc56953ac498582361f7e613cec38 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c5fb6bfd8ec4401fb0f9b3cf91ae5ef3 (127.19.194.3:43573), 39d31d4b04c14099bf89458586ff793c (127.19.194.1:37593)
I20260806 18:34:15.399978 22281 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:34:15.396564 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.194.3:43573 (local address 127.19.194.2:55193)
0806 18:34:15.396692 (+ 128us) negotiation.cc:107] Waiting for socket to connect
0806 18:34:15.396706 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0806 18:34:15.396800 (+ 94us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:34:15.397059 (+ 259us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:34:15.397062 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:34:15.397083 (+ 21us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 18:34:15.397203 (+ 120us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:34:15.397212 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:34:15.397998 (+ 786us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:34:15.398000 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:34:15.398958 (+ 958us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:34:15.398969 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:34:15.399676 (+ 707us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:34:15.399679 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:34:15.399755 (+ 76us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:34:15.399774 (+ 19us) client_negotiation.cc:769] Sending connection context
0806 18:34:15.399830 (+ 56us) client_negotiation.cc:240] Negotiation successful
0806 18:34:15.399881 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":46}
I20260806 18:34:15.400041 22554 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:34:15.396618 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.194.2:55193 (local address 127.19.194.3:43573)
0806 18:34:15.396756 (+ 138us) server_negotiation.cc:206] Beginning negotiation
0806 18:34:15.396762 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 18:34:15.396784 (+ 22us) server_negotiation.cc:407] Connection header received
0806 18:34:15.396887 (+ 103us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:34:15.396892 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:34:15.396943 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:34:15.397020 (+ 77us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 18:34:15.397299 (+ 279us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:34:15.397875 (+ 576us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:34:15.399082 (+ 1207us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:34:15.399534 (+ 452us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:34:15.399683 (+ 149us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:34:15.399732 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0806 18:34:15.399862 (+ 130us) server_negotiation.cc:299] Negotiation successful
0806 18:34:15.399929 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260806 18:34:15.400700 22488 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "51387960396d495ab763cdeee6fe1006" candidate_uuid: "b49bc56953ac498582361f7e613cec38" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c5fb6bfd8ec4401fb0f9b3cf91ae5ef3" is_pre_election: true
I20260806 18:34:15.400815 22557 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:34:15.397168 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.194.2:45481 (local address 127.19.194.1:37593)
0806 18:34:15.397278 (+ 110us) server_negotiation.cc:206] Beginning negotiation
0806 18:34:15.397283 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 18:34:15.397297 (+ 14us) server_negotiation.cc:407] Connection header received
0806 18:34:15.397335 (+ 38us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:34:15.397337 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:34:15.397377 (+ 40us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:34:15.397450 (+ 73us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 18:34:15.397958 (+ 508us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:34:15.398454 (+ 496us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:34:15.399640 (+ 1186us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:34:15.400261 (+ 621us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:34:15.400390 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:34:15.400440 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0806 18:34:15.400657 (+ 217us) server_negotiation.cc:299] Negotiation successful
0806 18:34:15.400714 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260806 18:34:15.400825 22567 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:34:15.396834 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.194.1:37593 (local address 127.19.194.2:45481)
0806 18:34:15.397153 (+ 319us) negotiation.cc:107] Waiting for socket to connect
0806 18:34:15.397169 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0806 18:34:15.397244 (+ 75us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:34:15.397453 (+ 209us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:34:15.397455 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:34:15.397466 (+ 11us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 18:34:15.397849 (+ 383us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:34:15.397854 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:34:15.398558 (+ 704us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:34:15.398561 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:34:15.399511 (+ 950us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:34:15.399520 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:34:15.400409 (+ 889us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:34:15.400412 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:34:15.400523 (+ 111us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:34:15.400547 (+ 24us) client_negotiation.cc:769] Sending connection context
0806 18:34:15.400622 (+ 75us) client_negotiation.cc:240] Negotiation successful
0806 18:34:15.400684 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":261,"thread_start_us":116,"threads_started":1}
I20260806 18:34:15.400957 22488 raft_consensus.cc:2454] T 51387960396d495ab763cdeee6fe1006 P c5fb6bfd8ec4401fb0f9b3cf91ae5ef3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b49bc56953ac498582361f7e613cec38 in term 0.
I20260806 18:34:15.401373 22274 leader_election.cc:302] T 51387960396d495ab763cdeee6fe1006 P b49bc56953ac498582361f7e613cec38 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b49bc56953ac498582361f7e613cec38, c5fb6bfd8ec4401fb0f9b3cf91ae5ef3; no voters:
I20260806 18:34:15.401645 22566 raft_consensus.cc:2781] T 51387960396d495ab763cdeee6fe1006 P b49bc56953ac498582361f7e613cec38 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 18:34:15.401633 22202 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "51387960396d495ab763cdeee6fe1006" candidate_uuid: "b49bc56953ac498582361f7e613cec38" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "39d31d4b04c14099bf89458586ff793c" is_pre_election: true
I20260806 18:34:15.401739 22566 raft_consensus.cc:482] T 51387960396d495ab763cdeee6fe1006 P b49bc56953ac498582361f7e613cec38 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 18:34:15.401786 22566 raft_consensus.cc:3037] T 51387960396d495ab763cdeee6fe1006 P b49bc56953ac498582361f7e613cec38 [term 0 FOLLOWER]: Advancing to term 1
I20260806 18:34:15.401890 22202 raft_consensus.cc:2454] T 51387960396d495ab763cdeee6fe1006 P 39d31d4b04c14099bf89458586ff793c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b49bc56953ac498582361f7e613cec38 in term 0.
I20260806 18:34:15.402922 22566 raft_consensus.cc:504] T 51387960396d495ab763cdeee6fe1006 P b49bc56953ac498582361f7e613cec38 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c5fb6bfd8ec4401fb0f9b3cf91ae5ef3" member_type: VOTER last_known_addr { host: "127.19.194.3" port: 43573 } } peers { permanent_uuid: "b49bc56953ac498582361f7e613cec38" member_type: VOTER last_known_addr { host: "127.19.194.2" port: 38833 } } peers { permanent_uuid: "39d31d4b04c14099bf89458586ff793c" member_type: VOTER last_known_addr { host: "127.19.194.1" port: 37593 } }
I20260806 18:34:15.403259 22566 leader_election.cc:288] T 51387960396d495ab763cdeee6fe1006 P b49bc56953ac498582361f7e613cec38 [CANDIDATE]: Term 1 election: Requested vote from peers c5fb6bfd8ec4401fb0f9b3cf91ae5ef3 (127.19.194.3:43573), 39d31d4b04c14099bf89458586ff793c (127.19.194.1:37593)
I20260806 18:34:15.403561 22488 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "51387960396d495ab763cdeee6fe1006" candidate_uuid: "b49bc56953ac498582361f7e613cec38" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c5fb6bfd8ec4401fb0f9b3cf91ae5ef3"
I20260806 18:34:15.403666 22202 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "51387960396d495ab763cdeee6fe1006" candidate_uuid: "b49bc56953ac498582361f7e613cec38" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "39d31d4b04c14099bf89458586ff793c"
I20260806 18:34:15.403695 22488 raft_consensus.cc:3037] T 51387960396d495ab763cdeee6fe1006 P c5fb6bfd8ec4401fb0f9b3cf91ae5ef3 [term 0 FOLLOWER]: Advancing to term 1
I20260806 18:34:15.403825 22202 raft_consensus.cc:3037] T 51387960396d495ab763cdeee6fe1006 P 39d31d4b04c14099bf89458586ff793c [term 0 FOLLOWER]: Advancing to term 1
I20260806 18:34:15.404713 22488 raft_consensus.cc:2454] T 51387960396d495ab763cdeee6fe1006 P c5fb6bfd8ec4401fb0f9b3cf91ae5ef3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b49bc56953ac498582361f7e613cec38 in term 1.
I20260806 18:34:15.404783 22202 raft_consensus.cc:2454] T 51387960396d495ab763cdeee6fe1006 P 39d31d4b04c14099bf89458586ff793c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b49bc56953ac498582361f7e613cec38 in term 1.
I20260806 18:34:15.405017 22274 leader_election.cc:302] T 51387960396d495ab763cdeee6fe1006 P b49bc56953ac498582361f7e613cec38 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b49bc56953ac498582361f7e613cec38, c5fb6bfd8ec4401fb0f9b3cf91ae5ef3; no voters:
I20260806 18:34:15.405282 22566 raft_consensus.cc:2781] T 51387960396d495ab763cdeee6fe1006 P b49bc56953ac498582361f7e613cec38 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 18:34:15.405495 22566 raft_consensus.cc:686] T 51387960396d495ab763cdeee6fe1006 P b49bc56953ac498582361f7e613cec38 [term 1 LEADER]: Becoming Leader. State: Replica: b49bc56953ac498582361f7e613cec38, State: Running, Role: LEADER
I20260806 18:34:15.405810 22566 consensus_queue.cc:237] T 51387960396d495ab763cdeee6fe1006 P b49bc56953ac498582361f7e613cec38 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c5fb6bfd8ec4401fb0f9b3cf91ae5ef3" member_type: VOTER last_known_addr { host: "127.19.194.3" port: 43573 } } peers { permanent_uuid: "b49bc56953ac498582361f7e613cec38" member_type: VOTER last_known_addr { host: "127.19.194.2" port: 38833 } } peers { permanent_uuid: "39d31d4b04c14099bf89458586ff793c" member_type: VOTER last_known_addr { host: "127.19.194.1" port: 37593 } }
I20260806 18:34:15.408687 22037 catalog_manager.cc:5495] T 51387960396d495ab763cdeee6fe1006 P b49bc56953ac498582361f7e613cec38 reported cstate change: term changed from 0 to 1, leader changed from <none> to b49bc56953ac498582361f7e613cec38 (127.19.194.2). New cstate: current_term: 1 leader_uuid: "b49bc56953ac498582361f7e613cec38" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c5fb6bfd8ec4401fb0f9b3cf91ae5ef3" member_type: VOTER last_known_addr { host: "127.19.194.3" port: 43573 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b49bc56953ac498582361f7e613cec38" member_type: VOTER last_known_addr { host: "127.19.194.2" port: 38833 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "39d31d4b04c14099bf89458586ff793c" member_type: VOTER last_known_addr { host: "127.19.194.1" port: 37593 } health_report { overall_health: UNKNOWN } } }
Aug 06 18:34:15 dist-test-slave-1xrh krb5kdc[21994](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041254, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.194.62@KRBTEST.COM for kudu/127.19.194.62@KRBTEST.COM
I20260806 18:34:15.444563 22090 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:34:15.436692 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40688 (local address 127.19.194.62:33617)
0806 18:34:15.436815 (+ 123us) server_negotiation.cc:206] Beginning negotiation
0806 18:34:15.436820 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 18:34:15.436953 (+ 133us) server_negotiation.cc:407] Connection header received
0806 18:34:15.437060 (+ 107us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:34:15.437063 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:34:15.437100 (+ 37us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:34:15.437167 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:34:15.437756 (+ 589us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:34:15.438251 (+ 495us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:34:15.438977 (+ 726us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:34:15.439230 (+ 253us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:34:15.439314 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:34:15.441465 (+ 2151us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:34:15.441468 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:34:15.441470 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 18:34:15.441492 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:34:15.442895 (+ 1403us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:34:15.443368 (+ 473us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:34:15.443370 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:34:15.443372 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:34:15.443444 (+ 72us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:34:15.443713 (+ 269us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:34:15.443715 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:34:15.443716 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:34:15.443875 (+ 159us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:34:15.443948 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0806 18:34:15.444243 (+ 295us) server_negotiation.cc:299] Negotiation successful
0806 18:34:15.444406 (+ 163us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":37,"server-negotiator.queue_time_us":50}
I20260806 18:34:15.444563 22578 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:34:15.436620 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.194.62:33617 (local address 127.0.0.1:40688)
0806 18:34:15.436874 (+ 254us) negotiation.cc:107] Waiting for socket to connect
0806 18:34:15.436886 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0806 18:34:15.436980 (+ 94us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:34:15.437198 (+ 218us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:34:15.437201 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:34:15.437571 (+ 370us) client_negotiation.cc:189] Negotiated authn=SASL
0806 18:34:15.437661 (+ 90us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:34:15.437667 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:34:15.438339 (+ 672us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:34:15.438342 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:34:15.438886 (+ 544us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:34:15.438893 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:34:15.439344 (+ 451us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:34:15.439346 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:34:15.439408 (+ 62us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:34:15.439968 (+ 560us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 18:34:15.439980 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0806 18:34:15.441330 (+ 1350us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 18:34:15.442982 (+ 1652us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:34:15.442990 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:34:15.442995 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:34:15.443296 (+ 301us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:34:15.443523 (+ 227us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:34:15.443525 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:34:15.443526 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:34:15.443603 (+ 77us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:34:15.443962 (+ 359us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 18:34:15.443967 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 18:34:15.444065 (+ 98us) client_negotiation.cc:769] Sending connection context
0806 18:34:15.444179 (+ 114us) client_negotiation.cc:240] Negotiation successful
0806 18:34:15.444349 (+ 170us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":182,"thread_start_us":94,"threads_started":1}
I20260806 18:34:15.448508 22037 catalog_manager.cc:1908] Servicing CreateTable request from {username='kudu', principal='kudu/127.19.194.62@KRBTEST.COM'} at 127.0.0.1:40688:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260806 18:34:15.449206 22037 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 18:34:15.453938 22324 tablet_service.cc:1467] Processing CreateTablet for tablet cb357a65da67405f98b536d9e4f218b6 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=6ecfaa0b7821464eadf0b0609609559a]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260806 18:34:15.453917 22180 tablet_service.cc:1467] Processing CreateTablet for tablet cb357a65da67405f98b536d9e4f218b6 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=6ecfaa0b7821464eadf0b0609609559a]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260806 18:34:15.454300 22468 tablet_service.cc:1467] Processing CreateTablet for tablet cb357a65da67405f98b536d9e4f218b6 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=6ecfaa0b7821464eadf0b0609609559a]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260806 18:34:15.454675 22324 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet cb357a65da67405f98b536d9e4f218b6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 18:34:15.454674 22180 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet cb357a65da67405f98b536d9e4f218b6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 18:34:15.454929 22468 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet cb357a65da67405f98b536d9e4f218b6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 18:34:15.457394 22558 tablet_bootstrap.cc:492] T cb357a65da67405f98b536d9e4f218b6 P c5fb6bfd8ec4401fb0f9b3cf91ae5ef3: Bootstrap starting.
I20260806 18:34:15.457548 22559 tablet_bootstrap.cc:492] T cb357a65da67405f98b536d9e4f218b6 P b49bc56953ac498582361f7e613cec38: Bootstrap starting.
I20260806 18:34:15.457592 22560 tablet_bootstrap.cc:492] T cb357a65da67405f98b536d9e4f218b6 P 39d31d4b04c14099bf89458586ff793c: Bootstrap starting.
I20260806 18:34:15.458390 22559 tablet_bootstrap.cc:654] T cb357a65da67405f98b536d9e4f218b6 P b49bc56953ac498582361f7e613cec38: Neither blocks nor log segments found. Creating new log.
I20260806 18:34:15.458437 22558 tablet_bootstrap.cc:654] T cb357a65da67405f98b536d9e4f218b6 P c5fb6bfd8ec4401fb0f9b3cf91ae5ef3: Neither blocks nor log segments found. Creating new log.
I20260806 18:34:15.458464 22560 tablet_bootstrap.cc:654] T cb357a65da67405f98b536d9e4f218b6 P 39d31d4b04c14099bf89458586ff793c: Neither blocks nor log segments found. Creating new log.
I20260806 18:34:15.459801 22559 tablet_bootstrap.cc:492] T cb357a65da67405f98b536d9e4f218b6 P b49bc56953ac498582361f7e613cec38: No bootstrap required, opened a new log
I20260806 18:34:15.459856 22558 tablet_bootstrap.cc:492] T cb357a65da67405f98b536d9e4f218b6 P c5fb6bfd8ec4401fb0f9b3cf91ae5ef3: No bootstrap required, opened a new log
I20260806 18:34:15.459899 22559 ts_tablet_manager.cc:1397] T cb357a65da67405f98b536d9e4f218b6 P b49bc56953ac498582361f7e613cec38: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260806 18:34:15.459949 22558 ts_tablet_manager.cc:1397] T cb357a65da67405f98b536d9e4f218b6 P c5fb6bfd8ec4401fb0f9b3cf91ae5ef3: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260806 18:34:15.460328 22558 raft_consensus.cc:348] T cb357a65da67405f98b536d9e4f218b6 P c5fb6bfd8ec4401fb0f9b3cf91ae5ef3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "39d31d4b04c14099bf89458586ff793c" member_type: VOTER last_known_addr { host: "127.19.194.1" port: 37593 } } peers { permanent_uuid: "b49bc56953ac498582361f7e613cec38" member_type: VOTER last_known_addr { host: "127.19.194.2" port: 38833 } } peers { permanent_uuid: "c5fb6bfd8ec4401fb0f9b3cf91ae5ef3" member_type: VOTER last_known_addr { host: "127.19.194.3" port: 43573 } }
I20260806 18:34:15.460451 22560 tablet_bootstrap.cc:492] T cb357a65da67405f98b536d9e4f218b6 P 39d31d4b04c14099bf89458586ff793c: No bootstrap required, opened a new log
I20260806 18:34:15.460443 22558 raft_consensus.cc:374] T cb357a65da67405f98b536d9e4f218b6 P c5fb6bfd8ec4401fb0f9b3cf91ae5ef3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 18:34:15.460393 22559 raft_consensus.cc:348] T cb357a65da67405f98b536d9e4f218b6 P b49bc56953ac498582361f7e613cec38 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "39d31d4b04c14099bf89458586ff793c" member_type: VOTER last_known_addr { host: "127.19.194.1" port: 37593 } } peers { permanent_uuid: "b49bc56953ac498582361f7e613cec38" member_type: VOTER last_known_addr { host: "127.19.194.2" port: 38833 } } peers { permanent_uuid: "c5fb6bfd8ec4401fb0f9b3cf91ae5ef3" member_type: VOTER last_known_addr { host: "127.19.194.3" port: 43573 } }
I20260806 18:34:15.460489 22558 raft_consensus.cc:729] T cb357a65da67405f98b536d9e4f218b6 P c5fb6bfd8ec4401fb0f9b3cf91ae5ef3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c5fb6bfd8ec4401fb0f9b3cf91ae5ef3, State: Initialized, Role: FOLLOWER
I20260806 18:34:15.460500 22559 raft_consensus.cc:374] T cb357a65da67405f98b536d9e4f218b6 P b49bc56953ac498582361f7e613cec38 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 18:34:15.460536 22560 ts_tablet_manager.cc:1397] T cb357a65da67405f98b536d9e4f218b6 P 39d31d4b04c14099bf89458586ff793c: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.000s
I20260806 18:34:15.460546 22559 raft_consensus.cc:729] T cb357a65da67405f98b536d9e4f218b6 P b49bc56953ac498582361f7e613cec38 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b49bc56953ac498582361f7e613cec38, State: Initialized, Role: FOLLOWER
I20260806 18:34:15.460632 22558 consensus_queue.cc:260] T cb357a65da67405f98b536d9e4f218b6 P c5fb6bfd8ec4401fb0f9b3cf91ae5ef3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "39d31d4b04c14099bf89458586ff793c" member_type: VOTER last_known_addr { host: "127.19.194.1" port: 37593 } } peers { permanent_uuid: "b49bc56953ac498582361f7e613cec38" member_type: VOTER last_known_addr { host: "127.19.194.2" port: 38833 } } peers { permanent_uuid: "c5fb6bfd8ec4401fb0f9b3cf91ae5ef3" member_type: VOTER last_known_addr { host: "127.19.194.3" port: 43573 } }
I20260806 18:34:15.460678 22559 consensus_queue.cc:260] T cb357a65da67405f98b536d9e4f218b6 P b49bc56953ac498582361f7e613cec38 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "39d31d4b04c14099bf89458586ff793c" member_type: VOTER last_known_addr { host: "127.19.194.1" port: 37593 } } peers { permanent_uuid: "b49bc56953ac498582361f7e613cec38" member_type: VOTER last_known_addr { host: "127.19.194.2" port: 38833 } } peers { permanent_uuid: "c5fb6bfd8ec4401fb0f9b3cf91ae5ef3" member_type: VOTER last_known_addr { host: "127.19.194.3" port: 43573 } }
I20260806 18:34:15.460935 22559 ts_tablet_manager.cc:1428] T cb357a65da67405f98b536d9e4f218b6 P b49bc56953ac498582361f7e613cec38: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260806 18:34:15.460981 22560 raft_consensus.cc:348] T cb357a65da67405f98b536d9e4f218b6 P 39d31d4b04c14099bf89458586ff793c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "39d31d4b04c14099bf89458586ff793c" member_type: VOTER last_known_addr { host: "127.19.194.1" port: 37593 } } peers { permanent_uuid: "b49bc56953ac498582361f7e613cec38" member_type: VOTER last_known_addr { host: "127.19.194.2" port: 38833 } } peers { permanent_uuid: "c5fb6bfd8ec4401fb0f9b3cf91ae5ef3" member_type: VOTER last_known_addr { host: "127.19.194.3" port: 43573 } }
I20260806 18:34:15.460979 22565 tablet_replica.cc:439] T cb357a65da67405f98b536d9e4f218b6 P c5fb6bfd8ec4401fb0f9b3cf91ae5ef3: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "39d31d4b04c14099bf89458586ff793c" member_type: VOTER last_known_addr { host: "127.19.194.1" port: 37593 } } peers { permanent_uuid: "b49bc56953ac498582361f7e613cec38" member_type: VOTER last_known_addr { host: "127.19.194.2" port: 38833 } } peers { permanent_uuid: "c5fb6bfd8ec4401fb0f9b3cf91ae5ef3" member_type: VOTER last_known_addr { host: "127.19.194.3" port: 43573 } } }
I20260806 18:34:15.461090 22560 raft_consensus.cc:374] T cb357a65da67405f98b536d9e4f218b6 P 39d31d4b04c14099bf89458586ff793c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 18:34:15.461120 22560 raft_consensus.cc:729] T cb357a65da67405f98b536d9e4f218b6 P 39d31d4b04c14099bf89458586ff793c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 39d31d4b04c14099bf89458586ff793c, State: Initialized, Role: FOLLOWER
I20260806 18:34:15.461122 22565 tablet_replica.cc:442] T cb357a65da67405f98b536d9e4f218b6 P c5fb6bfd8ec4401fb0f9b3cf91ae5ef3: This TxnStatusTablet replica's current role is: FOLLOWER
I20260806 18:34:15.461251 22560 consensus_queue.cc:260] T cb357a65da67405f98b536d9e4f218b6 P 39d31d4b04c14099bf89458586ff793c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "39d31d4b04c14099bf89458586ff793c" member_type: VOTER last_known_addr { host: "127.19.194.1" port: 37593 } } peers { permanent_uuid: "b49bc56953ac498582361f7e613cec38" member_type: VOTER last_known_addr { host: "127.19.194.2" port: 38833 } } peers { permanent_uuid: "c5fb6bfd8ec4401fb0f9b3cf91ae5ef3" member_type: VOTER last_known_addr { host: "127.19.194.3" port: 43573 } }
I20260806 18:34:15.460996 22558 ts_tablet_manager.cc:1428] T cb357a65da67405f98b536d9e4f218b6 P c5fb6bfd8ec4401fb0f9b3cf91ae5ef3: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260806 18:34:15.461541 22560 ts_tablet_manager.cc:1428] T cb357a65da67405f98b536d9e4f218b6 P 39d31d4b04c14099bf89458586ff793c: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260806 18:34:15.461506 22564 tablet_replica.cc:439] T cb357a65da67405f98b536d9e4f218b6 P 39d31d4b04c14099bf89458586ff793c: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "39d31d4b04c14099bf89458586ff793c" member_type: VOTER last_known_addr { host: "127.19.194.1" port: 37593 } } peers { permanent_uuid: "b49bc56953ac498582361f7e613cec38" member_type: VOTER last_known_addr { host: "127.19.194.2" port: 38833 } } peers { permanent_uuid: "c5fb6bfd8ec4401fb0f9b3cf91ae5ef3" member_type: VOTER last_known_addr { host: "127.19.194.3" port: 43573 } } }
I20260806 18:34:15.461638 22564 tablet_replica.cc:442] T cb357a65da67405f98b536d9e4f218b6 P 39d31d4b04c14099bf89458586ff793c: This TxnStatusTablet replica's current role is: FOLLOWER
I20260806 18:34:15.461731 22566 tablet_replica.cc:439] T cb357a65da67405f98b536d9e4f218b6 P b49bc56953ac498582361f7e613cec38: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "39d31d4b04c14099bf89458586ff793c" member_type: VOTER last_known_addr { host: "127.19.194.1" port: 37593 } } peers { permanent_uuid: "b49bc56953ac498582361f7e613cec38" member_type: VOTER last_known_addr { host: "127.19.194.2" port: 38833 } } peers { permanent_uuid: "c5fb6bfd8ec4401fb0f9b3cf91ae5ef3" member_type: VOTER last_known_addr { host: "127.19.194.3" port: 43573 } } }
I20260806 18:34:15.461849 22566 tablet_replica.cc:442] T cb357a65da67405f98b536d9e4f218b6 P b49bc56953ac498582361f7e613cec38: This TxnStatusTablet replica's current role is: FOLLOWER
I20260806 18:34:15.490955 22566 raft_consensus.cc:482] T cb357a65da67405f98b536d9e4f218b6 P b49bc56953ac498582361f7e613cec38 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 18:34:15.491071 22566 raft_consensus.cc:504] T cb357a65da67405f98b536d9e4f218b6 P b49bc56953ac498582361f7e613cec38 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "39d31d4b04c14099bf89458586ff793c" member_type: VOTER last_known_addr { host: "127.19.194.1" port: 37593 } } peers { permanent_uuid: "b49bc56953ac498582361f7e613cec38" member_type: VOTER last_known_addr { host: "127.19.194.2" port: 38833 } } peers { permanent_uuid: "c5fb6bfd8ec4401fb0f9b3cf91ae5ef3" member_type: VOTER last_known_addr { host: "127.19.194.3" port: 43573 } }
I20260806 18:34:15.491609 22566 leader_election.cc:288] T cb357a65da67405f98b536d9e4f218b6 P b49bc56953ac498582361f7e613cec38 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 39d31d4b04c14099bf89458586ff793c (127.19.194.1:37593), c5fb6bfd8ec4401fb0f9b3cf91ae5ef3 (127.19.194.3:43573)
I20260806 18:34:15.492022 22202 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cb357a65da67405f98b536d9e4f218b6" candidate_uuid: "b49bc56953ac498582361f7e613cec38" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "39d31d4b04c14099bf89458586ff793c" is_pre_election: true
I20260806 18:34:15.492116 22488 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cb357a65da67405f98b536d9e4f218b6" candidate_uuid: "b49bc56953ac498582361f7e613cec38" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c5fb6bfd8ec4401fb0f9b3cf91ae5ef3" is_pre_election: true
I20260806 18:34:15.492179 22202 raft_consensus.cc:2454] T cb357a65da67405f98b536d9e4f218b6 P 39d31d4b04c14099bf89458586ff793c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b49bc56953ac498582361f7e613cec38 in term 0.
I20260806 18:34:15.492252 22488 raft_consensus.cc:2454] T cb357a65da67405f98b536d9e4f218b6 P c5fb6bfd8ec4401fb0f9b3cf91ae5ef3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b49bc56953ac498582361f7e613cec38 in term 0.
I20260806 18:34:15.492488 22274 leader_election.cc:302] T cb357a65da67405f98b536d9e4f218b6 P b49bc56953ac498582361f7e613cec38 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 39d31d4b04c14099bf89458586ff793c, b49bc56953ac498582361f7e613cec38; no voters:
I20260806 18:34:15.492667 22566 raft_consensus.cc:2781] T cb357a65da67405f98b536d9e4f218b6 P b49bc56953ac498582361f7e613cec38 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 18:34:15.492741 22566 raft_consensus.cc:482] T cb357a65da67405f98b536d9e4f218b6 P b49bc56953ac498582361f7e613cec38 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 18:34:15.492796 22566 raft_consensus.cc:3037] T cb357a65da67405f98b536d9e4f218b6 P b49bc56953ac498582361f7e613cec38 [term 0 FOLLOWER]: Advancing to term 1
I20260806 18:34:15.493757 22566 raft_consensus.cc:504] T cb357a65da67405f98b536d9e4f218b6 P b49bc56953ac498582361f7e613cec38 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "39d31d4b04c14099bf89458586ff793c" member_type: VOTER last_known_addr { host: "127.19.194.1" port: 37593 } } peers { permanent_uuid: "b49bc56953ac498582361f7e613cec38" member_type: VOTER last_known_addr { host: "127.19.194.2" port: 38833 } } peers { permanent_uuid: "c5fb6bfd8ec4401fb0f9b3cf91ae5ef3" member_type: VOTER last_known_addr { host: "127.19.194.3" port: 43573 } }
I20260806 18:34:15.494071 22566 leader_election.cc:288] T cb357a65da67405f98b536d9e4f218b6 P b49bc56953ac498582361f7e613cec38 [CANDIDATE]: Term 1 election: Requested vote from peers 39d31d4b04c14099bf89458586ff793c (127.19.194.1:37593), c5fb6bfd8ec4401fb0f9b3cf91ae5ef3 (127.19.194.3:43573)
I20260806 18:34:15.494386 22202 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cb357a65da67405f98b536d9e4f218b6" candidate_uuid: "b49bc56953ac498582361f7e613cec38" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "39d31d4b04c14099bf89458586ff793c"
I20260806 18:34:15.494458 22488 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cb357a65da67405f98b536d9e4f218b6" candidate_uuid: "b49bc56953ac498582361f7e613cec38" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c5fb6bfd8ec4401fb0f9b3cf91ae5ef3"
I20260806 18:34:15.494517 22202 raft_consensus.cc:3037] T cb357a65da67405f98b536d9e4f218b6 P 39d31d4b04c14099bf89458586ff793c [term 0 FOLLOWER]: Advancing to term 1
I20260806 18:34:15.494557 22488 raft_consensus.cc:3037] T cb357a65da67405f98b536d9e4f218b6 P c5fb6bfd8ec4401fb0f9b3cf91ae5ef3 [term 0 FOLLOWER]: Advancing to term 1
I20260806 18:34:15.495715 22202 raft_consensus.cc:2454] T cb357a65da67405f98b536d9e4f218b6 P 39d31d4b04c14099bf89458586ff793c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b49bc56953ac498582361f7e613cec38 in term 1.
I20260806 18:34:15.495715 22488 raft_consensus.cc:2454] T cb357a65da67405f98b536d9e4f218b6 P c5fb6bfd8ec4401fb0f9b3cf91ae5ef3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b49bc56953ac498582361f7e613cec38 in term 1.
I20260806 18:34:15.495994 22274 leader_election.cc:302] T cb357a65da67405f98b536d9e4f218b6 P b49bc56953ac498582361f7e613cec38 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 39d31d4b04c14099bf89458586ff793c, b49bc56953ac498582361f7e613cec38; no voters:
I20260806 18:34:15.496188 22566 raft_consensus.cc:2781] T cb357a65da67405f98b536d9e4f218b6 P b49bc56953ac498582361f7e613cec38 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 18:34:15.496294 22566 raft_consensus.cc:686] T cb357a65da67405f98b536d9e4f218b6 P b49bc56953ac498582361f7e613cec38 [term 1 LEADER]: Becoming Leader. State: Replica: b49bc56953ac498582361f7e613cec38, State: Running, Role: LEADER
I20260806 18:34:15.496436 22566 consensus_queue.cc:237] T cb357a65da67405f98b536d9e4f218b6 P b49bc56953ac498582361f7e613cec38 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "39d31d4b04c14099bf89458586ff793c" member_type: VOTER last_known_addr { host: "127.19.194.1" port: 37593 } } peers { permanent_uuid: "b49bc56953ac498582361f7e613cec38" member_type: VOTER last_known_addr { host: "127.19.194.2" port: 38833 } } peers { permanent_uuid: "c5fb6bfd8ec4401fb0f9b3cf91ae5ef3" member_type: VOTER last_known_addr { host: "127.19.194.3" port: 43573 } }
I20260806 18:34:15.497185 22569 tablet_replica.cc:439] T cb357a65da67405f98b536d9e4f218b6 P b49bc56953ac498582361f7e613cec38: TxnStatusTablet state changed. Reason: New leader b49bc56953ac498582361f7e613cec38. Latest consensus state: current_term: 1 leader_uuid: "b49bc56953ac498582361f7e613cec38" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "39d31d4b04c14099bf89458586ff793c" member_type: VOTER last_known_addr { host: "127.19.194.1" port: 37593 } } peers { permanent_uuid: "b49bc56953ac498582361f7e613cec38" member_type: VOTER last_known_addr { host: "127.19.194.2" port: 38833 } } peers { permanent_uuid: "c5fb6bfd8ec4401fb0f9b3cf91ae5ef3" member_type: VOTER last_known_addr { host: "127.19.194.3" port: 43573 } } }
I20260806 18:34:15.497289 22569 tablet_replica.cc:442] T cb357a65da67405f98b536d9e4f218b6 P b49bc56953ac498582361f7e613cec38: This TxnStatusTablet replica's current role is: LEADER
I20260806 18:34:15.497612 22583 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260806 18:34:15.497856 22583 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260806 18:34:15.498178 22037 catalog_manager.cc:5495] T cb357a65da67405f98b536d9e4f218b6 P b49bc56953ac498582361f7e613cec38 reported cstate change: term changed from 0 to 1, leader changed from <none> to b49bc56953ac498582361f7e613cec38 (127.19.194.2). New cstate: current_term: 1 leader_uuid: "b49bc56953ac498582361f7e613cec38" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "39d31d4b04c14099bf89458586ff793c" member_type: VOTER last_known_addr { host: "127.19.194.1" port: 37593 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b49bc56953ac498582361f7e613cec38" member_type: VOTER last_known_addr { host: "127.19.194.2" port: 38833 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c5fb6bfd8ec4401fb0f9b3cf91ae5ef3" member_type: VOTER last_known_addr { host: "127.19.194.3" port: 43573 } health_report { overall_health: UNKNOWN } } }
I20260806 18:34:15.516311 22556 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:34:15.512733 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55096 (local address 127.19.194.2:38833)
0806 18:34:15.512853 (+ 120us) server_negotiation.cc:206] Beginning negotiation
0806 18:34:15.512858 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 18:34:15.512919 (+ 61us) server_negotiation.cc:407] Connection header received
0806 18:34:15.513127 (+ 208us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:34:15.513130 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:34:15.513180 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:34:15.513272 (+ 92us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 18:34:15.513519 (+ 247us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:34:15.514037 (+ 518us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:34:15.514876 (+ 839us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:34:15.515098 (+ 222us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:34:15.515266 (+ 168us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:34:15.515558 (+ 292us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 18:34:15.515910 (+ 352us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 18:34:15.516040 (+ 130us) server_negotiation.cc:1035] Waiting for connection context
0806 18:34:15.516122 (+ 82us) server_negotiation.cc:299] Negotiation successful
0806 18:34:15.516178 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260806 18:34:15.516311 22578 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:34:15.512659 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.194.2:38833 (local address 127.0.0.1:55096)
0806 18:34:15.512835 (+ 176us) negotiation.cc:107] Waiting for socket to connect
0806 18:34:15.512852 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0806 18:34:15.512990 (+ 138us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:34:15.513299 (+ 309us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:34:15.513302 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:34:15.513310 (+ 8us) client_negotiation.cc:189] Negotiated authn=TOKEN
0806 18:34:15.513417 (+ 107us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:34:15.513422 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:34:15.514148 (+ 726us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:34:15.514151 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:34:15.514752 (+ 601us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:34:15.514760 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:34:15.515266 (+ 506us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:34:15.515269 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:34:15.515369 (+ 100us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:34:15.515411 (+ 42us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0806 18:34:15.516027 (+ 616us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0806 18:34:15.516042 (+ 15us) client_negotiation.cc:769] Sending connection context
0806 18:34:15.516111 (+ 69us) client_negotiation.cc:240] Negotiation successful
0806 18:34:15.516193 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":53}
I20260806 18:34:15.521561 22488 raft_consensus.cc:1261] T cb357a65da67405f98b536d9e4f218b6 P c5fb6bfd8ec4401fb0f9b3cf91ae5ef3 [term 1 FOLLOWER]: Refusing update from remote peer b49bc56953ac498582361f7e613cec38: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 18:34:15.521581 22202 raft_consensus.cc:1261] T cb357a65da67405f98b536d9e4f218b6 P 39d31d4b04c14099bf89458586ff793c [term 1 FOLLOWER]: Refusing update from remote peer b49bc56953ac498582361f7e613cec38: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 18:34:15.522203 22566 consensus_queue.cc:1035] T cb357a65da67405f98b536d9e4f218b6 P b49bc56953ac498582361f7e613cec38 [LEADER]: Connected to new peer: Peer: permanent_uuid: "39d31d4b04c14099bf89458586ff793c" member_type: VOTER last_known_addr { host: "127.19.194.1" port: 37593 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 18:34:15.522387 22569 consensus_queue.cc:1035] T cb357a65da67405f98b536d9e4f218b6 P b49bc56953ac498582361f7e613cec38 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c5fb6bfd8ec4401fb0f9b3cf91ae5ef3" member_type: VOTER last_known_addr { host: "127.19.194.3" port: 43573 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 18:34:15.525918 22564 tablet_replica.cc:439] T cb357a65da67405f98b536d9e4f218b6 P 39d31d4b04c14099bf89458586ff793c: TxnStatusTablet state changed. Reason: New leader b49bc56953ac498582361f7e613cec38. Latest consensus state: current_term: 1 leader_uuid: "b49bc56953ac498582361f7e613cec38" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "39d31d4b04c14099bf89458586ff793c" member_type: VOTER last_known_addr { host: "127.19.194.1" port: 37593 } } peers { permanent_uuid: "b49bc56953ac498582361f7e613cec38" member_type: VOTER last_known_addr { host: "127.19.194.2" port: 38833 } } peers { permanent_uuid: "c5fb6bfd8ec4401fb0f9b3cf91ae5ef3" member_type: VOTER last_known_addr { host: "127.19.194.3" port: 43573 } } }
I20260806 18:34:15.526048 22564 tablet_replica.cc:442] T cb357a65da67405f98b536d9e4f218b6 P 39d31d4b04c14099bf89458586ff793c: This TxnStatusTablet replica's current role is: FOLLOWER
I20260806 18:34:15.526449 22565 tablet_replica.cc:439] T cb357a65da67405f98b536d9e4f218b6 P c5fb6bfd8ec4401fb0f9b3cf91ae5ef3: TxnStatusTablet state changed. Reason: New leader b49bc56953ac498582361f7e613cec38. Latest consensus state: current_term: 1 leader_uuid: "b49bc56953ac498582361f7e613cec38" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "39d31d4b04c14099bf89458586ff793c" member_type: VOTER last_known_addr { host: "127.19.194.1" port: 37593 } } peers { permanent_uuid: "b49bc56953ac498582361f7e613cec38" member_type: VOTER last_known_addr { host: "127.19.194.2" port: 38833 } } peers { permanent_uuid: "c5fb6bfd8ec4401fb0f9b3cf91ae5ef3" member_type: VOTER last_known_addr { host: "127.19.194.3" port: 43573 } } }
I20260806 18:34:15.526576 22565 tablet_replica.cc:442] T cb357a65da67405f98b536d9e4f218b6 P c5fb6bfd8ec4401fb0f9b3cf91ae5ef3: This TxnStatusTablet replica's current role is: FOLLOWER
I20260806 18:34:15.530297 22569 tablet_replica.cc:439] T cb357a65da67405f98b536d9e4f218b6 P b49bc56953ac498582361f7e613cec38: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "b49bc56953ac498582361f7e613cec38" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "39d31d4b04c14099bf89458586ff793c" member_type: VOTER last_known_addr { host: "127.19.194.1" port: 37593 } } peers { permanent_uuid: "b49bc56953ac498582361f7e613cec38" member_type: VOTER last_known_addr { host: "127.19.194.2" port: 38833 } } peers { permanent_uuid: "c5fb6bfd8ec4401fb0f9b3cf91ae5ef3" member_type: VOTER last_known_addr { host: "127.19.194.3" port: 43573 } } }
I20260806 18:34:15.530427 22569 tablet_replica.cc:442] T cb357a65da67405f98b536d9e4f218b6 P b49bc56953ac498582361f7e613cec38: This TxnStatusTablet replica's current role is: LEADER
I20260806 18:34:15.531534 22566 tablet_replica.cc:439] T cb357a65da67405f98b536d9e4f218b6 P b49bc56953ac498582361f7e613cec38: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "b49bc56953ac498582361f7e613cec38" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "39d31d4b04c14099bf89458586ff793c" member_type: VOTER last_known_addr { host: "127.19.194.1" port: 37593 } } peers { permanent_uuid: "b49bc56953ac498582361f7e613cec38" member_type: VOTER last_known_addr { host: "127.19.194.2" port: 38833 } } peers { permanent_uuid: "c5fb6bfd8ec4401fb0f9b3cf91ae5ef3" member_type: VOTER last_known_addr { host: "127.19.194.3" port: 43573 } } }
I20260806 18:34:15.531662 22566 tablet_replica.cc:442] T cb357a65da67405f98b536d9e4f218b6 P b49bc56953ac498582361f7e613cec38: This TxnStatusTablet replica's current role is: LEADER
I20260806 18:34:15.532091 22564 tablet_replica.cc:439] T cb357a65da67405f98b536d9e4f218b6 P 39d31d4b04c14099bf89458586ff793c: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "b49bc56953ac498582361f7e613cec38" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "39d31d4b04c14099bf89458586ff793c" member_type: VOTER last_known_addr { host: "127.19.194.1" port: 37593 } } peers { permanent_uuid: "b49bc56953ac498582361f7e613cec38" member_type: VOTER last_known_addr { host: "127.19.194.2" port: 38833 } } peers { permanent_uuid: "c5fb6bfd8ec4401fb0f9b3cf91ae5ef3" member_type: VOTER last_known_addr { host: "127.19.194.3" port: 43573 } } }
I20260806 18:34:15.532207 22564 tablet_replica.cc:442] T cb357a65da67405f98b536d9e4f218b6 P 39d31d4b04c14099bf89458586ff793c: This TxnStatusTablet replica's current role is: FOLLOWER
I20260806 18:34:15.533106 22565 tablet_replica.cc:439] T cb357a65da67405f98b536d9e4f218b6 P c5fb6bfd8ec4401fb0f9b3cf91ae5ef3: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "b49bc56953ac498582361f7e613cec38" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "39d31d4b04c14099bf89458586ff793c" member_type: VOTER last_known_addr { host: "127.19.194.1" port: 37593 } } peers { permanent_uuid: "b49bc56953ac498582361f7e613cec38" member_type: VOTER last_known_addr { host: "127.19.194.2" port: 38833 } } peers { permanent_uuid: "c5fb6bfd8ec4401fb0f9b3cf91ae5ef3" member_type: VOTER last_known_addr { host: "127.19.194.3" port: 43573 } } }
I20260806 18:34:15.533214 22565 tablet_replica.cc:442] T cb357a65da67405f98b536d9e4f218b6 P c5fb6bfd8ec4401fb0f9b3cf91ae5ef3: This TxnStatusTablet replica's current role is: FOLLOWER
I20260806 18:34:15.533838 22587 mvcc.cc:205] Tried to move back new op lower bound from 7315624982610038784 to 7315624982514905088. Current Snapshot: MvccSnapshot[applied={T|T < 7315624982610038784 or (T in {7315624982610038784})}]
I20260806 18:34:15.534775 22588 mvcc.cc:205] Tried to move back new op lower bound from 7315624982610038784 to 7315624982514905088. Current Snapshot: MvccSnapshot[applied={T|T < 7315624982610038784}]
I20260806 18:34:15.542126 22556 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:34:15.538733 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55106 (local address 127.19.194.2:38833)
0806 18:34:15.538858 (+ 125us) server_negotiation.cc:206] Beginning negotiation
0806 18:34:15.538862 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 18:34:15.538878 (+ 16us) server_negotiation.cc:407] Connection header received
0806 18:34:15.539001 (+ 123us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:34:15.539004 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:34:15.539058 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:34:15.539166 (+ 108us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 18:34:15.539413 (+ 247us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:34:15.540122 (+ 709us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:34:15.541021 (+ 899us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:34:15.541353 (+ 332us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:34:15.541454 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:34:15.541700 (+ 246us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 18:34:15.541779 (+ 79us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 18:34:15.541851 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0806 18:34:15.541930 (+ 79us) server_negotiation.cc:299] Negotiation successful
0806 18:34:15.541982 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
W20260806 18:34:15.553059 22535 tablet.cc:2369] T 51387960396d495ab763cdeee6fe1006 P c5fb6bfd8ec4401fb0f9b3cf91ae5ef3: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 18:34:15.553165 22398 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:34:15.549676 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.194.2:38833 (local address 127.19.194.2:38917)
0806 18:34:15.549888 (+ 212us) negotiation.cc:107] Waiting for socket to connect
0806 18:34:15.549902 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0806 18:34:15.549991 (+ 89us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:34:15.550210 (+ 219us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:34:15.550213 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:34:15.550220 (+ 7us) client_negotiation.cc:189] Negotiated authn=TOKEN
0806 18:34:15.550328 (+ 108us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:34:15.550333 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:34:15.550970 (+ 637us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:34:15.550973 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:34:15.551959 (+ 986us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:34:15.551969 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:34:15.552371 (+ 402us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:34:15.552374 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:34:15.552467 (+ 93us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:34:15.552535 (+ 68us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0806 18:34:15.552821 (+ 286us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0806 18:34:15.552829 (+ 8us) client_negotiation.cc:769] Sending connection context
0806 18:34:15.552890 (+ 61us) client_negotiation.cc:240] Negotiation successful
0806 18:34:15.552965 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":113}
I20260806 18:34:15.553170 22556 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:34:15.549742 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.194.2:38917 (local address 127.19.194.2:38833)
0806 18:34:15.549845 (+ 103us) server_negotiation.cc:206] Beginning negotiation
0806 18:34:15.549848 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 18:34:15.549956 (+ 108us) server_negotiation.cc:407] Connection header received
0806 18:34:15.550082 (+ 126us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:34:15.550084 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:34:15.550121 (+ 37us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:34:15.550180 (+ 59us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 18:34:15.550420 (+ 240us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:34:15.550873 (+ 453us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:34:15.552080 (+ 1207us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:34:15.552269 (+ 189us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:34:15.552339 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:34:15.552679 (+ 340us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 18:34:15.552739 (+ 60us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 18:34:15.552789 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0806 18:34:15.552941 (+ 152us) server_negotiation.cc:299] Negotiation successful
0806 18:34:15.552997 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34}
I20260806 18:34:15.559170 22202 raft_consensus.cc:1261] T 51387960396d495ab763cdeee6fe1006 P 39d31d4b04c14099bf89458586ff793c [term 1 FOLLOWER]: Refusing update from remote peer b49bc56953ac498582361f7e613cec38: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 18:34:15.559249 22488 raft_consensus.cc:1261] T 51387960396d495ab763cdeee6fe1006 P c5fb6bfd8ec4401fb0f9b3cf91ae5ef3 [term 1 FOLLOWER]: Refusing update from remote peer b49bc56953ac498582361f7e613cec38: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 18:34:15.559551 22569 consensus_queue.cc:1035] T 51387960396d495ab763cdeee6fe1006 P b49bc56953ac498582361f7e613cec38 [LEADER]: Connected to new peer: Peer: permanent_uuid: "39d31d4b04c14099bf89458586ff793c" member_type: VOTER last_known_addr { host: "127.19.194.1" port: 37593 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 18:34:15.559710 22566 consensus_queue.cc:1035] T 51387960396d495ab763cdeee6fe1006 P b49bc56953ac498582361f7e613cec38 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c5fb6bfd8ec4401fb0f9b3cf91ae5ef3" member_type: VOTER last_known_addr { host: "127.19.194.3" port: 43573 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 18:34:15.563941 22586 mvcc.cc:205] Tried to move back new op lower bound from 7315624982767214592 to 7315624982145167360. Current Snapshot: MvccSnapshot[applied={T|T < 7315624982767214592}]
I20260806 18:34:15.600221 22040 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:40686:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 18:34:15.600410 22040 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:40686:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 18:34:15.602694 22040 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 805a3be867fa43bdafe4c8974a803d51: Sending DeleteTablet for 3 replicas of tablet 51387960396d495ab763cdeee6fe1006
I20260806 18:34:15.603434 22324 tablet_service.cc:1514] Processing DeleteTablet for tablet 51387960396d495ab763cdeee6fe1006 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 18:34:15 UTC) from {username='kudu', principal='kudu/127.19.194.62@KRBTEST.COM'} at 127.0.0.1:55086
I20260806 18:34:15.603569 22180 tablet_service.cc:1514] Processing DeleteTablet for tablet 51387960396d495ab763cdeee6fe1006 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 18:34:15 UTC) from {username='kudu', principal='kudu/127.19.194.62@KRBTEST.COM'} at 127.0.0.1:49642
I20260806 18:34:15.604094 22603 tablet_replica.cc:330] T 51387960396d495ab763cdeee6fe1006 P 39d31d4b04c14099bf89458586ff793c: stopping tablet replica
I20260806 18:34:15.604029 22468 tablet_service.cc:1514] Processing DeleteTablet for tablet 51387960396d495ab763cdeee6fe1006 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 18:34:15 UTC) from {username='kudu', principal='kudu/127.19.194.62@KRBTEST.COM'} at 127.0.0.1:36884
I20260806 18:34:15.604424 22603 raft_consensus.cc:2229] T 51387960396d495ab763cdeee6fe1006 P 39d31d4b04c14099bf89458586ff793c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 18:34:15.604496 22604 tablet_replica.cc:330] T 51387960396d495ab763cdeee6fe1006 P c5fb6bfd8ec4401fb0f9b3cf91ae5ef3: stopping tablet replica
I20260806 18:34:15.604660 22603 raft_consensus.cc:2258] T 51387960396d495ab763cdeee6fe1006 P 39d31d4b04c14099bf89458586ff793c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 18:34:15.604755 22604 raft_consensus.cc:2229] T 51387960396d495ab763cdeee6fe1006 P c5fb6bfd8ec4401fb0f9b3cf91ae5ef3 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 18:34:15.604776 22040 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:40686:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260806 18:34:15.605034 22602 tablet_replica.cc:330] T 51387960396d495ab763cdeee6fe1006 P b49bc56953ac498582361f7e613cec38: stopping tablet replica
W20260806 18:34:15.605151 22040 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 18:34:15.605273 22602 raft_consensus.cc:2229] T 51387960396d495ab763cdeee6fe1006 P b49bc56953ac498582361f7e613cec38 [term 1 LEADER]: Raft consensus shutting down.
I20260806 18:34:15.605624 22602 raft_consensus.cc:2258] T 51387960396d495ab763cdeee6fe1006 P b49bc56953ac498582361f7e613cec38 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 18:34:15.605865 22604 raft_consensus.cc:2258] T 51387960396d495ab763cdeee6fe1006 P c5fb6bfd8ec4401fb0f9b3cf91ae5ef3 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 18:34:15.606319 22603 ts_tablet_manager.cc:1905] T 51387960396d495ab763cdeee6fe1006 P 39d31d4b04c14099bf89458586ff793c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 18:34:15.607342 22604 ts_tablet_manager.cc:1905] T 51387960396d495ab763cdeee6fe1006 P c5fb6bfd8ec4401fb0f9b3cf91ae5ef3: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 18:34:15.609452 22603 ts_tablet_manager.cc:1918] T 51387960396d495ab763cdeee6fe1006 P 39d31d4b04c14099bf89458586ff793c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260806 18:34:15.609539 22603 log.cc:1196] T 51387960396d495ab763cdeee6fe1006 P 39d31d4b04c14099bf89458586ff793c: Deleting WAL directory at /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786041246094603-20232-0/minicluster-data/ts-0/wal/wals/51387960396d495ab763cdeee6fe1006
I20260806 18:34:15.609850 22603 ts_tablet_manager.cc:1939] T 51387960396d495ab763cdeee6fe1006 P 39d31d4b04c14099bf89458586ff793c: Deleting consensus metadata
I20260806 18:34:15.610390 22604 ts_tablet_manager.cc:1918] T 51387960396d495ab763cdeee6fe1006 P c5fb6bfd8ec4401fb0f9b3cf91ae5ef3: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260806 18:34:15.610476 22604 log.cc:1196] T 51387960396d495ab763cdeee6fe1006 P c5fb6bfd8ec4401fb0f9b3cf91ae5ef3: Deleting WAL directory at /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786041246094603-20232-0/minicluster-data/ts-2/wal/wals/51387960396d495ab763cdeee6fe1006
I20260806 18:34:15.610783 22604 ts_tablet_manager.cc:1939] T 51387960396d495ab763cdeee6fe1006 P c5fb6bfd8ec4401fb0f9b3cf91ae5ef3: Deleting consensus metadata
I20260806 18:34:15.611151 22024 catalog_manager.cc:4841] TS 39d31d4b04c14099bf89458586ff793c (127.19.194.1:37593): tablet 51387960396d495ab763cdeee6fe1006 (table test-table [id=1aff44fa39374b29a01198a905d82f93]) successfully deleted
I20260806 18:34:15.611928 22024 catalog_manager.cc:4841] TS c5fb6bfd8ec4401fb0f9b3cf91ae5ef3 (127.19.194.3:43573): tablet 51387960396d495ab763cdeee6fe1006 (table test-table [id=1aff44fa39374b29a01198a905d82f93]) successfully deleted
I20260806 18:34:15.612386 22602 ts_tablet_manager.cc:1905] T 51387960396d495ab763cdeee6fe1006 P b49bc56953ac498582361f7e613cec38: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 18:34:15.614514 22602 ts_tablet_manager.cc:1918] T 51387960396d495ab763cdeee6fe1006 P b49bc56953ac498582361f7e613cec38: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260806 18:34:15.614581 22602 log.cc:1196] T 51387960396d495ab763cdeee6fe1006 P b49bc56953ac498582361f7e613cec38: Deleting WAL directory at /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786041246094603-20232-0/minicluster-data/ts-1/wal/wals/51387960396d495ab763cdeee6fe1006
I20260806 18:34:15.614794 22602 ts_tablet_manager.cc:1939] T 51387960396d495ab763cdeee6fe1006 P b49bc56953ac498582361f7e613cec38: Deleting consensus metadata
I20260806 18:34:15.615794 22025 catalog_manager.cc:4841] TS b49bc56953ac498582361f7e613cec38 (127.19.194.2:38833): tablet 51387960396d495ab763cdeee6fe1006 (table test-table [id=1aff44fa39374b29a01198a905d82f93]) successfully deleted
I20260806 18:34:15.618801 22180 tablet_service.cc:1467] Processing CreateTablet for tablet d64ff13a430b46dcbd0f04fcd4739cd2 (DEFAULT_TABLE table=test-table [id=7f24b18c5357407eab26ca6920641e6f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 18:34:15.619068 22180 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d64ff13a430b46dcbd0f04fcd4739cd2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 18:34:15.619613 22324 tablet_service.cc:1467] Processing CreateTablet for tablet d64ff13a430b46dcbd0f04fcd4739cd2 (DEFAULT_TABLE table=test-table [id=7f24b18c5357407eab26ca6920641e6f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 18:34:15.619879 22324 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d64ff13a430b46dcbd0f04fcd4739cd2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 18:34:15.621503 22559 tablet_bootstrap.cc:492] T d64ff13a430b46dcbd0f04fcd4739cd2 P b49bc56953ac498582361f7e613cec38: Bootstrap starting.
I20260806 18:34:15.622161 22468 tablet_service.cc:1467] Processing CreateTablet for tablet d64ff13a430b46dcbd0f04fcd4739cd2 (DEFAULT_TABLE table=test-table [id=7f24b18c5357407eab26ca6920641e6f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 18:34:15.622401 22559 tablet_bootstrap.cc:654] T d64ff13a430b46dcbd0f04fcd4739cd2 P b49bc56953ac498582361f7e613cec38: Neither blocks nor log segments found. Creating new log.
I20260806 18:34:15.622435 22468 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d64ff13a430b46dcbd0f04fcd4739cd2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 18:34:15.624769 22558 tablet_bootstrap.cc:492] T d64ff13a430b46dcbd0f04fcd4739cd2 P c5fb6bfd8ec4401fb0f9b3cf91ae5ef3: Bootstrap starting.
I20260806 18:34:15.625494 22560 tablet_bootstrap.cc:492] T d64ff13a430b46dcbd0f04fcd4739cd2 P 39d31d4b04c14099bf89458586ff793c: Bootstrap starting.
I20260806 18:34:15.625718 22558 tablet_bootstrap.cc:654] T d64ff13a430b46dcbd0f04fcd4739cd2 P c5fb6bfd8ec4401fb0f9b3cf91ae5ef3: Neither blocks nor log segments found. Creating new log.
I20260806 18:34:15.626428 22560 tablet_bootstrap.cc:654] T d64ff13a430b46dcbd0f04fcd4739cd2 P 39d31d4b04c14099bf89458586ff793c: Neither blocks nor log segments found. Creating new log.
I20260806 18:34:15.626513 22559 tablet_bootstrap.cc:492] T d64ff13a430b46dcbd0f04fcd4739cd2 P b49bc56953ac498582361f7e613cec38: No bootstrap required, opened a new log
I20260806 18:34:15.626595 22559 ts_tablet_manager.cc:1397] T d64ff13a430b46dcbd0f04fcd4739cd2 P b49bc56953ac498582361f7e613cec38: Time spent bootstrapping tablet: real 0.005s user 0.002s sys 0.000s
I20260806 18:34:15.627074 22559 raft_consensus.cc:348] T d64ff13a430b46dcbd0f04fcd4739cd2 P b49bc56953ac498582361f7e613cec38 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "39d31d4b04c14099bf89458586ff793c" member_type: VOTER last_known_addr { host: "127.19.194.1" port: 37593 } } peers { permanent_uuid: "b49bc56953ac498582361f7e613cec38" member_type: VOTER last_known_addr { host: "127.19.194.2" port: 38833 } } peers { permanent_uuid: "c5fb6bfd8ec4401fb0f9b3cf91ae5ef3" member_type: VOTER last_known_addr { host: "127.19.194.3" port: 43573 } }
I20260806 18:34:15.627231 22559 raft_consensus.cc:374] T d64ff13a430b46dcbd0f04fcd4739cd2 P b49bc56953ac498582361f7e613cec38 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 18:34:15.627277 22559 raft_consensus.cc:729] T d64ff13a430b46dcbd0f04fcd4739cd2 P b49bc56953ac498582361f7e613cec38 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b49bc56953ac498582361f7e613cec38, State: Initialized, Role: FOLLOWER
I20260806 18:34:15.627411 22559 consensus_queue.cc:260] T d64ff13a430b46dcbd0f04fcd4739cd2 P b49bc56953ac498582361f7e613cec38 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "39d31d4b04c14099bf89458586ff793c" member_type: VOTER last_known_addr { host: "127.19.194.1" port: 37593 } } peers { permanent_uuid: "b49bc56953ac498582361f7e613cec38" member_type: VOTER last_known_addr { host: "127.19.194.2" port: 38833 } } peers { permanent_uuid: "c5fb6bfd8ec4401fb0f9b3cf91ae5ef3" member_type: VOTER last_known_addr { host: "127.19.194.3" port: 43573 } }
I20260806 18:34:15.627732 22559 ts_tablet_manager.cc:1428] T d64ff13a430b46dcbd0f04fcd4739cd2 P b49bc56953ac498582361f7e613cec38: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260806 18:34:15.627801 22558 tablet_bootstrap.cc:492] T d64ff13a430b46dcbd0f04fcd4739cd2 P c5fb6bfd8ec4401fb0f9b3cf91ae5ef3: No bootstrap required, opened a new log
I20260806 18:34:15.627876 22558 ts_tablet_manager.cc:1397] T d64ff13a430b46dcbd0f04fcd4739cd2 P c5fb6bfd8ec4401fb0f9b3cf91ae5ef3: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.000s
I20260806 18:34:15.628654 22558 raft_consensus.cc:348] T d64ff13a430b46dcbd0f04fcd4739cd2 P c5fb6bfd8ec4401fb0f9b3cf91ae5ef3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "39d31d4b04c14099bf89458586ff793c" member_type: VOTER last_known_addr { host: "127.19.194.1" port: 37593 } } peers { permanent_uuid: "b49bc56953ac498582361f7e613cec38" member_type: VOTER last_known_addr { host: "127.19.194.2" port: 38833 } } peers { permanent_uuid: "c5fb6bfd8ec4401fb0f9b3cf91ae5ef3" member_type: VOTER last_known_addr { host: "127.19.194.3" port: 43573 } }
I20260806 18:34:15.628793 22558 raft_consensus.cc:374] T d64ff13a430b46dcbd0f04fcd4739cd2 P c5fb6bfd8ec4401fb0f9b3cf91ae5ef3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 18:34:15.628826 22558 raft_consensus.cc:729] T d64ff13a430b46dcbd0f04fcd4739cd2 P c5fb6bfd8ec4401fb0f9b3cf91ae5ef3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c5fb6bfd8ec4401fb0f9b3cf91ae5ef3, State: Initialized, Role: FOLLOWER
I20260806 18:34:15.628962 22558 consensus_queue.cc:260] T d64ff13a430b46dcbd0f04fcd4739cd2 P c5fb6bfd8ec4401fb0f9b3cf91ae5ef3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "39d31d4b04c14099bf89458586ff793c" member_type: VOTER last_known_addr { host: "127.19.194.1" port: 37593 } } peers { permanent_uuid: "b49bc56953ac498582361f7e613cec38" member_type: VOTER last_known_addr { host: "127.19.194.2" port: 38833 } } peers { permanent_uuid: "c5fb6bfd8ec4401fb0f9b3cf91ae5ef3" member_type: VOTER last_known_addr { host: "127.19.194.3" port: 43573 } }
I20260806 18:34:15.629240 22558 ts_tablet_manager.cc:1428] T d64ff13a430b46dcbd0f04fcd4739cd2 P c5fb6bfd8ec4401fb0f9b3cf91ae5ef3: Time spent starting tablet: real 0.001s user 0.001s sys 0.001s
I20260806 18:34:15.630848 22560 tablet_bootstrap.cc:492] T d64ff13a430b46dcbd0f04fcd4739cd2 P 39d31d4b04c14099bf89458586ff793c: No bootstrap required, opened a new log
I20260806 18:34:15.630936 22560 ts_tablet_manager.cc:1397] T d64ff13a430b46dcbd0f04fcd4739cd2 P 39d31d4b04c14099bf89458586ff793c: Time spent bootstrapping tablet: real 0.005s user 0.001s sys 0.001s
I20260806 18:34:15.631378 22560 raft_consensus.cc:348] T d64ff13a430b46dcbd0f04fcd4739cd2 P 39d31d4b04c14099bf89458586ff793c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "39d31d4b04c14099bf89458586ff793c" member_type: VOTER last_known_addr { host: "127.19.194.1" port: 37593 } } peers { permanent_uuid: "b49bc56953ac498582361f7e613cec38" member_type: VOTER last_known_addr { host: "127.19.194.2" port: 38833 } } peers { permanent_uuid: "c5fb6bfd8ec4401fb0f9b3cf91ae5ef3" member_type: VOTER last_known_addr { host: "127.19.194.3" port: 43573 } }
I20260806 18:34:15.631489 22560 raft_consensus.cc:374] T d64ff13a430b46dcbd0f04fcd4739cd2 P 39d31d4b04c14099bf89458586ff793c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 18:34:15.631532 22560 raft_consensus.cc:729] T d64ff13a430b46dcbd0f04fcd4739cd2 P 39d31d4b04c14099bf89458586ff793c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 39d31d4b04c14099bf89458586ff793c, State: Initialized, Role: FOLLOWER
I20260806 18:34:15.631669 22560 consensus_queue.cc:260] T d64ff13a430b46dcbd0f04fcd4739cd2 P 39d31d4b04c14099bf89458586ff793c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "39d31d4b04c14099bf89458586ff793c" member_type: VOTER last_known_addr { host: "127.19.194.1" port: 37593 } } peers { permanent_uuid: "b49bc56953ac498582361f7e613cec38" member_type: VOTER last_known_addr { host: "127.19.194.2" port: 38833 } } peers { permanent_uuid: "c5fb6bfd8ec4401fb0f9b3cf91ae5ef3" member_type: VOTER last_known_addr { host: "127.19.194.3" port: 43573 } }
I20260806 18:34:15.631951 22560 ts_tablet_manager.cc:1428] T d64ff13a430b46dcbd0f04fcd4739cd2 P 39d31d4b04c14099bf89458586ff793c: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20260806 18:34:15.774477 22564 raft_consensus.cc:482] T d64ff13a430b46dcbd0f04fcd4739cd2 P 39d31d4b04c14099bf89458586ff793c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 18:34:15.774653 22564 raft_consensus.cc:504] T d64ff13a430b46dcbd0f04fcd4739cd2 P 39d31d4b04c14099bf89458586ff793c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "39d31d4b04c14099bf89458586ff793c" member_type: VOTER last_known_addr { host: "127.19.194.1" port: 37593 } } peers { permanent_uuid: "b49bc56953ac498582361f7e613cec38" member_type: VOTER last_known_addr { host: "127.19.194.2" port: 38833 } } peers { permanent_uuid: "c5fb6bfd8ec4401fb0f9b3cf91ae5ef3" member_type: VOTER last_known_addr { host: "127.19.194.3" port: 43573 } }
I20260806 18:34:15.776144 22564 leader_election.cc:288] T d64ff13a430b46dcbd0f04fcd4739cd2 P 39d31d4b04c14099bf89458586ff793c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b49bc56953ac498582361f7e613cec38 (127.19.194.2:38833), c5fb6bfd8ec4401fb0f9b3cf91ae5ef3 (127.19.194.3:43573)
I20260806 18:34:15.780910 22554 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:34:15.776879 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.194.1:50467 (local address 127.19.194.3:43573)
0806 18:34:15.776997 (+ 118us) server_negotiation.cc:206] Beginning negotiation
0806 18:34:15.777002 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 18:34:15.777086 (+ 84us) server_negotiation.cc:407] Connection header received
0806 18:34:15.777492 (+ 406us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:34:15.777496 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:34:15.777561 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:34:15.777631 (+ 70us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 18:34:15.777891 (+ 260us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:34:15.778642 (+ 751us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:34:15.779911 (+ 1269us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:34:15.780293 (+ 382us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:34:15.780431 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:34:15.780488 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0806 18:34:15.780735 (+ 247us) server_negotiation.cc:299] Negotiation successful
0806 18:34:15.780808 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260806 18:34:15.780920 22609 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:34:15.776483 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.194.3:43573 (local address 127.19.194.1:50467)
0806 18:34:15.777000 (+ 517us) negotiation.cc:107] Waiting for socket to connect
0806 18:34:15.777012 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0806 18:34:15.777088 (+ 76us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:34:15.777648 (+ 560us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:34:15.777650 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:34:15.777668 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 18:34:15.777771 (+ 103us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:34:15.777777 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:34:15.778768 (+ 991us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:34:15.778771 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:34:15.779805 (+ 1034us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:34:15.779813 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:34:15.780474 (+ 661us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:34:15.780478 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:34:15.780576 (+ 98us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:34:15.780594 (+ 18us) client_negotiation.cc:769] Sending connection context
0806 18:34:15.780694 (+ 100us) client_negotiation.cc:240] Negotiation successful
0806 18:34:15.780767 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":467,"thread_start_us":109,"threads_started":1}
I20260806 18:34:15.781080 22556 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:34:15.776613 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.194.1:52051 (local address 127.19.194.2:38833)
0806 18:34:15.776751 (+ 138us) server_negotiation.cc:206] Beginning negotiation
0806 18:34:15.776756 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 18:34:15.776989 (+ 233us) server_negotiation.cc:407] Connection header received
0806 18:34:15.777180 (+ 191us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:34:15.777183 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:34:15.777236 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:34:15.777331 (+ 95us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 18:34:15.777580 (+ 249us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:34:15.778178 (+ 598us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:34:15.779640 (+ 1462us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:34:15.780267 (+ 627us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:34:15.780426 (+ 159us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:34:15.780481 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0806 18:34:15.780872 (+ 391us) server_negotiation.cc:299] Negotiation successful
0806 18:34:15.780958 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260806 18:34:15.781471 22608 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:34:15.776354 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.194.2:38833 (local address 127.19.194.1:52051)
0806 18:34:15.776895 (+ 541us) negotiation.cc:107] Waiting for socket to connect
0806 18:34:15.776909 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0806 18:34:15.777079 (+ 170us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:34:15.777335 (+ 256us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:34:15.777337 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:34:15.777349 (+ 12us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 18:34:15.777451 (+ 102us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:34:15.777454 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:34:15.778292 (+ 838us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:34:15.778295 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:34:15.779501 (+ 1206us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:34:15.779513 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:34:15.780548 (+ 1035us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:34:15.780552 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:34:15.780717 (+ 165us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:34:15.780742 (+ 25us) client_negotiation.cc:769] Sending connection context
0806 18:34:15.781251 (+ 509us) client_negotiation.cc:240] Negotiation successful
0806 18:34:15.781338 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":469,"thread_start_us":107,"threads_started":1}
I20260806 18:34:15.781770 22487 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d64ff13a430b46dcbd0f04fcd4739cd2" candidate_uuid: "39d31d4b04c14099bf89458586ff793c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c5fb6bfd8ec4401fb0f9b3cf91ae5ef3" is_pre_election: true
I20260806 18:34:15.781955 22487 raft_consensus.cc:2454] T d64ff13a430b46dcbd0f04fcd4739cd2 P c5fb6bfd8ec4401fb0f9b3cf91ae5ef3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 39d31d4b04c14099bf89458586ff793c in term 0.
I20260806 18:34:15.782356 22130 leader_election.cc:302] T d64ff13a430b46dcbd0f04fcd4739cd2 P 39d31d4b04c14099bf89458586ff793c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 39d31d4b04c14099bf89458586ff793c, c5fb6bfd8ec4401fb0f9b3cf91ae5ef3; no voters:
I20260806 18:34:15.782501 22344 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d64ff13a430b46dcbd0f04fcd4739cd2" candidate_uuid: "39d31d4b04c14099bf89458586ff793c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b49bc56953ac498582361f7e613cec38" is_pre_election: true
I20260806 18:34:15.782644 22564 raft_consensus.cc:2781] T d64ff13a430b46dcbd0f04fcd4739cd2 P 39d31d4b04c14099bf89458586ff793c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 18:34:15.782706 22564 raft_consensus.cc:482] T d64ff13a430b46dcbd0f04fcd4739cd2 P 39d31d4b04c14099bf89458586ff793c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 18:34:15.782745 22564 raft_consensus.cc:3037] T d64ff13a430b46dcbd0f04fcd4739cd2 P 39d31d4b04c14099bf89458586ff793c [term 0 FOLLOWER]: Advancing to term 1
I20260806 18:34:15.782734 22344 raft_consensus.cc:2454] T d64ff13a430b46dcbd0f04fcd4739cd2 P b49bc56953ac498582361f7e613cec38 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 39d31d4b04c14099bf89458586ff793c in term 0.
I20260806 18:34:15.784153 22564 raft_consensus.cc:504] T d64ff13a430b46dcbd0f04fcd4739cd2 P 39d31d4b04c14099bf89458586ff793c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "39d31d4b04c14099bf89458586ff793c" member_type: VOTER last_known_addr { host: "127.19.194.1" port: 37593 } } peers { permanent_uuid: "b49bc56953ac498582361f7e613cec38" member_type: VOTER last_known_addr { host: "127.19.194.2" port: 38833 } } peers { permanent_uuid: "c5fb6bfd8ec4401fb0f9b3cf91ae5ef3" member_type: VOTER last_known_addr { host: "127.19.194.3" port: 43573 } }
I20260806 18:34:15.784533 22564 leader_election.cc:288] T d64ff13a430b46dcbd0f04fcd4739cd2 P 39d31d4b04c14099bf89458586ff793c [CANDIDATE]: Term 1 election: Requested vote from peers b49bc56953ac498582361f7e613cec38 (127.19.194.2:38833), c5fb6bfd8ec4401fb0f9b3cf91ae5ef3 (127.19.194.3:43573)
I20260806 18:34:15.784866 22487 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d64ff13a430b46dcbd0f04fcd4739cd2" candidate_uuid: "39d31d4b04c14099bf89458586ff793c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c5fb6bfd8ec4401fb0f9b3cf91ae5ef3"
I20260806 18:34:15.784874 22344 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d64ff13a430b46dcbd0f04fcd4739cd2" candidate_uuid: "39d31d4b04c14099bf89458586ff793c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b49bc56953ac498582361f7e613cec38"
I20260806 18:34:15.784992 22344 raft_consensus.cc:3037] T d64ff13a430b46dcbd0f04fcd4739cd2 P b49bc56953ac498582361f7e613cec38 [term 0 FOLLOWER]: Advancing to term 1
I20260806 18:34:15.785022 22487 raft_consensus.cc:3037] T d64ff13a430b46dcbd0f04fcd4739cd2 P c5fb6bfd8ec4401fb0f9b3cf91ae5ef3 [term 0 FOLLOWER]: Advancing to term 1
I20260806 18:34:15.786091 22487 raft_consensus.cc:2454] T d64ff13a430b46dcbd0f04fcd4739cd2 P c5fb6bfd8ec4401fb0f9b3cf91ae5ef3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 39d31d4b04c14099bf89458586ff793c in term 1.
I20260806 18:34:15.786281 22344 raft_consensus.cc:2454] T d64ff13a430b46dcbd0f04fcd4739cd2 P b49bc56953ac498582361f7e613cec38 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 39d31d4b04c14099bf89458586ff793c in term 1.
I20260806 18:34:15.786418 22130 leader_election.cc:302] T d64ff13a430b46dcbd0f04fcd4739cd2 P 39d31d4b04c14099bf89458586ff793c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 39d31d4b04c14099bf89458586ff793c, c5fb6bfd8ec4401fb0f9b3cf91ae5ef3; no voters:
I20260806 18:34:15.786753 22564 raft_consensus.cc:2781] T d64ff13a430b46dcbd0f04fcd4739cd2 P 39d31d4b04c14099bf89458586ff793c [term 1 FOLLOWER]: Leader election won for term 1
I20260806 18:34:15.786995 22564 raft_consensus.cc:686] T d64ff13a430b46dcbd0f04fcd4739cd2 P 39d31d4b04c14099bf89458586ff793c [term 1 LEADER]: Becoming Leader. State: Replica: 39d31d4b04c14099bf89458586ff793c, State: Running, Role: LEADER
I20260806 18:34:15.787272 22564 consensus_queue.cc:237] T d64ff13a430b46dcbd0f04fcd4739cd2 P 39d31d4b04c14099bf89458586ff793c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "39d31d4b04c14099bf89458586ff793c" member_type: VOTER last_known_addr { host: "127.19.194.1" port: 37593 } } peers { permanent_uuid: "b49bc56953ac498582361f7e613cec38" member_type: VOTER last_known_addr { host: "127.19.194.2" port: 38833 } } peers { permanent_uuid: "c5fb6bfd8ec4401fb0f9b3cf91ae5ef3" member_type: VOTER last_known_addr { host: "127.19.194.3" port: 43573 } }
I20260806 18:34:15.789927 22037 catalog_manager.cc:5495] T d64ff13a430b46dcbd0f04fcd4739cd2 P 39d31d4b04c14099bf89458586ff793c reported cstate change: term changed from 0 to 1, leader changed from <none> to 39d31d4b04c14099bf89458586ff793c (127.19.194.1). New cstate: current_term: 1 leader_uuid: "39d31d4b04c14099bf89458586ff793c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "39d31d4b04c14099bf89458586ff793c" member_type: VOTER last_known_addr { host: "127.19.194.1" port: 37593 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b49bc56953ac498582361f7e613cec38" member_type: VOTER last_known_addr { host: "127.19.194.2" port: 38833 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c5fb6bfd8ec4401fb0f9b3cf91ae5ef3" member_type: VOTER last_known_addr { host: "127.19.194.3" port: 43573 } health_report { overall_health: UNKNOWN } } }
I20260806 18:34:15.826249 22557 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:34:15.822735 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49658 (local address 127.19.194.1:37593)
0806 18:34:15.822912 (+ 177us) server_negotiation.cc:206] Beginning negotiation
0806 18:34:15.822916 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 18:34:15.822930 (+ 14us) server_negotiation.cc:407] Connection header received
0806 18:34:15.823030 (+ 100us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:34:15.823032 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:34:15.823084 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:34:15.823222 (+ 138us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 18:34:15.823441 (+ 219us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:34:15.823997 (+ 556us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:34:15.824758 (+ 761us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:34:15.825008 (+ 250us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:34:15.825090 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:34:15.825394 (+ 304us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 18:34:15.825876 (+ 482us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 18:34:15.825967 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0806 18:34:15.826022 (+ 55us) server_negotiation.cc:299] Negotiation successful
0806 18:34:15.826067 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":90}
W20260806 18:34:15.832823 22393 tablet.cc:2369] T d64ff13a430b46dcbd0f04fcd4739cd2 P b49bc56953ac498582361f7e613cec38: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 18:34:15.836095 22556 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:34:15.832955 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.194.1:36123 (local address 127.19.194.2:38833)
0806 18:34:15.833064 (+ 109us) server_negotiation.cc:206] Beginning negotiation
0806 18:34:15.833069 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 18:34:15.833287 (+ 218us) server_negotiation.cc:407] Connection header received
0806 18:34:15.833381 (+ 94us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:34:15.833384 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:34:15.833430 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:34:15.833493 (+ 63us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 18:34:15.833694 (+ 201us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:34:15.834209 (+ 515us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:34:15.835076 (+ 867us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:34:15.835374 (+ 298us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:34:15.835510 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:34:15.835759 (+ 249us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 18:34:15.835825 (+ 66us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 18:34:15.835882 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0806 18:34:15.835952 (+ 70us) server_negotiation.cc:299] Negotiation successful
0806 18:34:15.835995 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260806 18:34:15.836144 22614 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:34:15.832852 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.194.2:38833 (local address 127.19.194.1:36123)
0806 18:34:15.833203 (+ 351us) negotiation.cc:107] Waiting for socket to connect
0806 18:34:15.833215 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0806 18:34:15.833298 (+ 83us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:34:15.833512 (+ 214us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:34:15.833514 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:34:15.833520 (+ 6us) client_negotiation.cc:189] Negotiated authn=TOKEN
0806 18:34:15.833612 (+ 92us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:34:15.833616 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:34:15.834320 (+ 704us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:34:15.834324 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:34:15.834954 (+ 630us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:34:15.834971 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:34:15.835523 (+ 552us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:34:15.835526 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:34:15.835596 (+ 70us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:34:15.835638 (+ 42us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0806 18:34:15.835895 (+ 257us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0806 18:34:15.835900 (+ 5us) client_negotiation.cc:769] Sending connection context
0806 18:34:15.835946 (+ 46us) client_negotiation.cc:240] Negotiation successful
0806 18:34:15.836009 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":270,"thread_start_us":97,"threads_started":1}
I20260806 18:34:15.841503 22487 raft_consensus.cc:1261] T d64ff13a430b46dcbd0f04fcd4739cd2 P c5fb6bfd8ec4401fb0f9b3cf91ae5ef3 [term 1 FOLLOWER]: Refusing update from remote peer 39d31d4b04c14099bf89458586ff793c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 18:34:15.841851 22344 raft_consensus.cc:1261] T d64ff13a430b46dcbd0f04fcd4739cd2 P b49bc56953ac498582361f7e613cec38 [term 1 FOLLOWER]: Refusing update from remote peer 39d31d4b04c14099bf89458586ff793c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 18:34:15.841964 22610 consensus_queue.cc:1035] T d64ff13a430b46dcbd0f04fcd4739cd2 P 39d31d4b04c14099bf89458586ff793c [LEADER]: Connected to new peer: Peer: permanent_uuid: "c5fb6bfd8ec4401fb0f9b3cf91ae5ef3" member_type: VOTER last_known_addr { host: "127.19.194.3" port: 43573 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 18:34:15.842381 22564 consensus_queue.cc:1035] T d64ff13a430b46dcbd0f04fcd4739cd2 P 39d31d4b04c14099bf89458586ff793c [LEADER]: Connected to new peer: Peer: permanent_uuid: "b49bc56953ac498582361f7e613cec38" member_type: VOTER last_known_addr { host: "127.19.194.2" port: 38833 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 18:34:15.859987 22557 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:34:15.856810 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.194.2:34357 (local address 127.19.194.1:37593)
0806 18:34:15.856924 (+ 114us) server_negotiation.cc:206] Beginning negotiation
0806 18:34:15.856929 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 18:34:15.856949 (+ 20us) server_negotiation.cc:407] Connection header received
0806 18:34:15.857081 (+ 132us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:34:15.857084 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:34:15.857129 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:34:15.857204 (+ 75us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 18:34:15.857449 (+ 245us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:34:15.857999 (+ 550us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:34:15.858709 (+ 710us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:34:15.858966 (+ 257us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:34:15.859049 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:34:15.859362 (+ 313us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 18:34:15.859445 (+ 83us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 18:34:15.859546 (+ 101us) server_negotiation.cc:1035] Waiting for connection context
0806 18:34:15.859802 (+ 256us) server_negotiation.cc:299] Negotiation successful
0806 18:34:15.859855 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260806 18:34:15.859990 22398 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:34:15.856762 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.194.1:37593 (local address 127.19.194.2:34357)
0806 18:34:15.856871 (+ 109us) negotiation.cc:107] Waiting for socket to connect
0806 18:34:15.856885 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0806 18:34:15.856987 (+ 102us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:34:15.857223 (+ 236us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:34:15.857226 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:34:15.857233 (+ 7us) client_negotiation.cc:189] Negotiated authn=TOKEN
0806 18:34:15.857340 (+ 107us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:34:15.857346 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:34:15.858084 (+ 738us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:34:15.858086 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:34:15.858599 (+ 513us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:34:15.858606 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:34:15.859087 (+ 481us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:34:15.859091 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:34:15.859196 (+ 105us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:34:15.859227 (+ 31us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0806 18:34:15.859667 (+ 440us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0806 18:34:15.859674 (+ 7us) client_negotiation.cc:769] Sending connection context
0806 18:34:15.859786 (+ 112us) client_negotiation.cc:240] Negotiation successful
0806 18:34:15.859867 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":26}
W20260806 18:34:15.860726 22587 tablet_replica.cc:1298] Aborted: operation has been aborted: cancelling pending write operations
I20260806 18:34:17.415601 22037 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:40686:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 18:34:17.415750 22037 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:40686:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 18:34:17.417981 22037 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 805a3be867fa43bdafe4c8974a803d51: Sending DeleteTablet for 3 replicas of tablet d64ff13a430b46dcbd0f04fcd4739cd2
I20260806 18:34:17.418558 22468 tablet_service.cc:1514] Processing DeleteTablet for tablet d64ff13a430b46dcbd0f04fcd4739cd2 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 18:34:17 UTC) from {username='kudu', principal='kudu/127.19.194.62@KRBTEST.COM'} at 127.0.0.1:36884
I20260806 18:34:17.418542 22180 tablet_service.cc:1514] Processing DeleteTablet for tablet d64ff13a430b46dcbd0f04fcd4739cd2 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 18:34:17 UTC) from {username='kudu', principal='kudu/127.19.194.62@KRBTEST.COM'} at 127.0.0.1:49642
I20260806 18:34:17.418553 22324 tablet_service.cc:1514] Processing DeleteTablet for tablet d64ff13a430b46dcbd0f04fcd4739cd2 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 18:34:17 UTC) from {username='kudu', principal='kudu/127.19.194.62@KRBTEST.COM'} at 127.0.0.1:55086
I20260806 18:34:17.418872 22641 tablet_replica.cc:330] T d64ff13a430b46dcbd0f04fcd4739cd2 P 39d31d4b04c14099bf89458586ff793c: stopping tablet replica
I20260806 18:34:17.418942 22642 tablet_replica.cc:330] T d64ff13a430b46dcbd0f04fcd4739cd2 P c5fb6bfd8ec4401fb0f9b3cf91ae5ef3: stopping tablet replica
I20260806 18:34:17.418999 22641 raft_consensus.cc:2229] T d64ff13a430b46dcbd0f04fcd4739cd2 P 39d31d4b04c14099bf89458586ff793c [term 1 LEADER]: Raft consensus shutting down.
I20260806 18:34:17.419068 22642 raft_consensus.cc:2229] T d64ff13a430b46dcbd0f04fcd4739cd2 P c5fb6bfd8ec4401fb0f9b3cf91ae5ef3 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 18:34:17.419207 22642 raft_consensus.cc:2258] T d64ff13a430b46dcbd0f04fcd4739cd2 P c5fb6bfd8ec4401fb0f9b3cf91ae5ef3 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 18:34:17.419329 22641 raft_consensus.cc:2258] T d64ff13a430b46dcbd0f04fcd4739cd2 P 39d31d4b04c14099bf89458586ff793c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 18:34:17.420038 22641 ts_tablet_manager.cc:1905] T d64ff13a430b46dcbd0f04fcd4739cd2 P 39d31d4b04c14099bf89458586ff793c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 18:34:17.422856 22642 ts_tablet_manager.cc:1905] T d64ff13a430b46dcbd0f04fcd4739cd2 P c5fb6bfd8ec4401fb0f9b3cf91ae5ef3: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 18:34:17.422960 22641 ts_tablet_manager.cc:1918] T d64ff13a430b46dcbd0f04fcd4739cd2 P 39d31d4b04c14099bf89458586ff793c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260806 18:34:17.423064 22641 log.cc:1196] T d64ff13a430b46dcbd0f04fcd4739cd2 P 39d31d4b04c14099bf89458586ff793c: Deleting WAL directory at /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786041246094603-20232-0/minicluster-data/ts-0/wal/wals/d64ff13a430b46dcbd0f04fcd4739cd2
I20260806 18:34:17.423447 22641 ts_tablet_manager.cc:1939] T d64ff13a430b46dcbd0f04fcd4739cd2 P 39d31d4b04c14099bf89458586ff793c: Deleting consensus metadata
I20260806 18:34:17.424099 22024 catalog_manager.cc:4841] TS 39d31d4b04c14099bf89458586ff793c (127.19.194.1:37593): tablet d64ff13a430b46dcbd0f04fcd4739cd2 (table test-table [id=7f24b18c5357407eab26ca6920641e6f]) successfully deleted
I20260806 18:34:17.425113 22642 ts_tablet_manager.cc:1918] T d64ff13a430b46dcbd0f04fcd4739cd2 P c5fb6bfd8ec4401fb0f9b3cf91ae5ef3: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260806 18:34:17.425206 22642 log.cc:1196] T d64ff13a430b46dcbd0f04fcd4739cd2 P c5fb6bfd8ec4401fb0f9b3cf91ae5ef3: Deleting WAL directory at /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786041246094603-20232-0/minicluster-data/ts-2/wal/wals/d64ff13a430b46dcbd0f04fcd4739cd2
I20260806 18:34:17.425446 22642 ts_tablet_manager.cc:1939] T d64ff13a430b46dcbd0f04fcd4739cd2 P c5fb6bfd8ec4401fb0f9b3cf91ae5ef3: Deleting consensus metadata
I20260806 18:34:17.426124 22643 tablet_replica.cc:330] T d64ff13a430b46dcbd0f04fcd4739cd2 P b49bc56953ac498582361f7e613cec38: stopping tablet replica
I20260806 18:34:17.426275 22643 raft_consensus.cc:2229] T d64ff13a430b46dcbd0f04fcd4739cd2 P b49bc56953ac498582361f7e613cec38 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 18:34:17.426266 22024 catalog_manager.cc:4841] TS c5fb6bfd8ec4401fb0f9b3cf91ae5ef3 (127.19.194.3:43573): tablet d64ff13a430b46dcbd0f04fcd4739cd2 (table test-table [id=7f24b18c5357407eab26ca6920641e6f]) successfully deleted
I20260806 18:34:17.426399 22643 raft_consensus.cc:2258] T d64ff13a430b46dcbd0f04fcd4739cd2 P b49bc56953ac498582361f7e613cec38 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 18:34:17.426949 22643 ts_tablet_manager.cc:1905] T d64ff13a430b46dcbd0f04fcd4739cd2 P b49bc56953ac498582361f7e613cec38: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 18:34:17.429296 22643 ts_tablet_manager.cc:1918] T d64ff13a430b46dcbd0f04fcd4739cd2 P b49bc56953ac498582361f7e613cec38: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260806 18:34:17.429378 22643 log.cc:1196] T d64ff13a430b46dcbd0f04fcd4739cd2 P b49bc56953ac498582361f7e613cec38: Deleting WAL directory at /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786041246094603-20232-0/minicluster-data/ts-1/wal/wals/d64ff13a430b46dcbd0f04fcd4739cd2
I20260806 18:34:17.429625 22643 ts_tablet_manager.cc:1939] T d64ff13a430b46dcbd0f04fcd4739cd2 P b49bc56953ac498582361f7e613cec38: Deleting consensus metadata
I20260806 18:34:17.430629 22025 catalog_manager.cc:4841] TS b49bc56953ac498582361f7e613cec38 (127.19.194.2:38833): tablet d64ff13a430b46dcbd0f04fcd4739cd2 (table test-table [id=7f24b18c5357407eab26ca6920641e6f]) successfully deleted
Aug 06 18:34:17 dist-test-slave-1xrh krb5kdc[21994](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041257, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 06 18:34:17 dist-test-slave-1xrh krb5kdc[21994](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041257, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.19.194.62@KRBTEST.COM
I20260806 18:34:17.452497 22651 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:34:17.439240 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40694 (local address 127.19.194.62:33617)
0806 18:34:17.439517 (+ 277us) server_negotiation.cc:206] Beginning negotiation
0806 18:34:17.439531 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0806 18:34:17.439544 (+ 13us) server_negotiation.cc:407] Connection header received
0806 18:34:17.439665 (+ 121us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:34:17.439667 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:34:17.439710 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:34:17.439785 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:34:17.440416 (+ 631us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:34:17.440954 (+ 538us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:34:17.441720 (+ 766us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:34:17.441950 (+ 230us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:34:17.442025 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:34:17.449446 (+ 7421us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:34:17.449449 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:34:17.449451 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 18:34:17.449475 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:34:17.450971 (+ 1496us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:34:17.451377 (+ 406us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:34:17.451380 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:34:17.451381 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:34:17.451458 (+ 77us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:34:17.451654 (+ 196us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:34:17.451655 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:34:17.451656 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:34:17.451812 (+ 156us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:34:17.451905 (+ 93us) server_negotiation.cc:1035] Waiting for connection context
0806 18:34:17.452120 (+ 215us) server_negotiation.cc:299] Negotiation successful
0806 18:34:17.452248 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":200,"thread_start_us":100,"threads_started":1}
I20260806 18:34:17.454828 22037 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:40694:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260806 18:34:17.455296 22037 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 18:34:17.459599 22180 tablet_service.cc:1467] Processing CreateTablet for tablet 5b6f5f19d21e48c68ed4bb9d04975f23 (DEFAULT_TABLE table=test-table [id=1ceddd20b4b143528573da4291425588]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 18:34:17.459614 22324 tablet_service.cc:1467] Processing CreateTablet for tablet 5b6f5f19d21e48c68ed4bb9d04975f23 (DEFAULT_TABLE table=test-table [id=1ceddd20b4b143528573da4291425588]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 18:34:17.459725 22468 tablet_service.cc:1467] Processing CreateTablet for tablet 5b6f5f19d21e48c68ed4bb9d04975f23 (DEFAULT_TABLE table=test-table [id=1ceddd20b4b143528573da4291425588]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 18:34:17.459889 22324 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5b6f5f19d21e48c68ed4bb9d04975f23. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 18:34:17.459890 22180 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5b6f5f19d21e48c68ed4bb9d04975f23. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 18:34:17.459955 22468 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5b6f5f19d21e48c68ed4bb9d04975f23. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 18:34:17.462051 22652 tablet_bootstrap.cc:492] T 5b6f5f19d21e48c68ed4bb9d04975f23 P b49bc56953ac498582361f7e613cec38: Bootstrap starting.
I20260806 18:34:17.462471 22653 tablet_bootstrap.cc:492] T 5b6f5f19d21e48c68ed4bb9d04975f23 P 39d31d4b04c14099bf89458586ff793c: Bootstrap starting.
I20260806 18:34:17.462980 22652 tablet_bootstrap.cc:654] T 5b6f5f19d21e48c68ed4bb9d04975f23 P b49bc56953ac498582361f7e613cec38: Neither blocks nor log segments found. Creating new log.
I20260806 18:34:17.463376 22653 tablet_bootstrap.cc:654] T 5b6f5f19d21e48c68ed4bb9d04975f23 P 39d31d4b04c14099bf89458586ff793c: Neither blocks nor log segments found. Creating new log.
I20260806 18:34:17.463390 22654 tablet_bootstrap.cc:492] T 5b6f5f19d21e48c68ed4bb9d04975f23 P c5fb6bfd8ec4401fb0f9b3cf91ae5ef3: Bootstrap starting.
I20260806 18:34:17.464385 22654 tablet_bootstrap.cc:654] T 5b6f5f19d21e48c68ed4bb9d04975f23 P c5fb6bfd8ec4401fb0f9b3cf91ae5ef3: Neither blocks nor log segments found. Creating new log.
I20260806 18:34:17.464552 22652 tablet_bootstrap.cc:492] T 5b6f5f19d21e48c68ed4bb9d04975f23 P b49bc56953ac498582361f7e613cec38: No bootstrap required, opened a new log
I20260806 18:34:17.464552 22653 tablet_bootstrap.cc:492] T 5b6f5f19d21e48c68ed4bb9d04975f23 P 39d31d4b04c14099bf89458586ff793c: No bootstrap required, opened a new log
I20260806 18:34:17.464629 22653 ts_tablet_manager.cc:1397] T 5b6f5f19d21e48c68ed4bb9d04975f23 P 39d31d4b04c14099bf89458586ff793c: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260806 18:34:17.464629 22652 ts_tablet_manager.cc:1397] T 5b6f5f19d21e48c68ed4bb9d04975f23 P b49bc56953ac498582361f7e613cec38: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260806 18:34:17.465031 22652 raft_consensus.cc:348] T 5b6f5f19d21e48c68ed4bb9d04975f23 P b49bc56953ac498582361f7e613cec38 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "39d31d4b04c14099bf89458586ff793c" member_type: VOTER last_known_addr { host: "127.19.194.1" port: 37593 } } peers { permanent_uuid: "b49bc56953ac498582361f7e613cec38" member_type: VOTER last_known_addr { host: "127.19.194.2" port: 38833 } } peers { permanent_uuid: "c5fb6bfd8ec4401fb0f9b3cf91ae5ef3" member_type: VOTER last_known_addr { host: "127.19.194.3" port: 43573 } }
I20260806 18:34:17.465030 22653 raft_consensus.cc:348] T 5b6f5f19d21e48c68ed4bb9d04975f23 P 39d31d4b04c14099bf89458586ff793c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "39d31d4b04c14099bf89458586ff793c" member_type: VOTER last_known_addr { host: "127.19.194.1" port: 37593 } } peers { permanent_uuid: "b49bc56953ac498582361f7e613cec38" member_type: VOTER last_known_addr { host: "127.19.194.2" port: 38833 } } peers { permanent_uuid: "c5fb6bfd8ec4401fb0f9b3cf91ae5ef3" member_type: VOTER last_known_addr { host: "127.19.194.3" port: 43573 } }
I20260806 18:34:17.465137 22652 raft_consensus.cc:374] T 5b6f5f19d21e48c68ed4bb9d04975f23 P b49bc56953ac498582361f7e613cec38 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 18:34:17.465137 22653 raft_consensus.cc:374] T 5b6f5f19d21e48c68ed4bb9d04975f23 P 39d31d4b04c14099bf89458586ff793c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 18:34:17.465173 22653 raft_consensus.cc:729] T 5b6f5f19d21e48c68ed4bb9d04975f23 P 39d31d4b04c14099bf89458586ff793c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 39d31d4b04c14099bf89458586ff793c, State: Initialized, Role: FOLLOWER
I20260806 18:34:17.465173 22652 raft_consensus.cc:729] T 5b6f5f19d21e48c68ed4bb9d04975f23 P b49bc56953ac498582361f7e613cec38 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b49bc56953ac498582361f7e613cec38, State: Initialized, Role: FOLLOWER
I20260806 18:34:17.465304 22653 consensus_queue.cc:260] T 5b6f5f19d21e48c68ed4bb9d04975f23 P 39d31d4b04c14099bf89458586ff793c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "39d31d4b04c14099bf89458586ff793c" member_type: VOTER last_known_addr { host: "127.19.194.1" port: 37593 } } peers { permanent_uuid: "b49bc56953ac498582361f7e613cec38" member_type: VOTER last_known_addr { host: "127.19.194.2" port: 38833 } } peers { permanent_uuid: "c5fb6bfd8ec4401fb0f9b3cf91ae5ef3" member_type: VOTER last_known_addr { host: "127.19.194.3" port: 43573 } }
I20260806 18:34:17.465304 22652 consensus_queue.cc:260] T 5b6f5f19d21e48c68ed4bb9d04975f23 P b49bc56953ac498582361f7e613cec38 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "39d31d4b04c14099bf89458586ff793c" member_type: VOTER last_known_addr { host: "127.19.194.1" port: 37593 } } peers { permanent_uuid: "b49bc56953ac498582361f7e613cec38" member_type: VOTER last_known_addr { host: "127.19.194.2" port: 38833 } } peers { permanent_uuid: "c5fb6bfd8ec4401fb0f9b3cf91ae5ef3" member_type: VOTER last_known_addr { host: "127.19.194.3" port: 43573 } }
I20260806 18:34:17.465565 22653 ts_tablet_manager.cc:1428] T 5b6f5f19d21e48c68ed4bb9d04975f23 P 39d31d4b04c14099bf89458586ff793c: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260806 18:34:17.465565 22652 ts_tablet_manager.cc:1428] T 5b6f5f19d21e48c68ed4bb9d04975f23 P b49bc56953ac498582361f7e613cec38: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260806 18:34:17.465612 22654 tablet_bootstrap.cc:492] T 5b6f5f19d21e48c68ed4bb9d04975f23 P c5fb6bfd8ec4401fb0f9b3cf91ae5ef3: No bootstrap required, opened a new log
I20260806 18:34:17.465682 22654 ts_tablet_manager.cc:1397] T 5b6f5f19d21e48c68ed4bb9d04975f23 P c5fb6bfd8ec4401fb0f9b3cf91ae5ef3: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260806 18:34:17.466131 22654 raft_consensus.cc:348] T 5b6f5f19d21e48c68ed4bb9d04975f23 P c5fb6bfd8ec4401fb0f9b3cf91ae5ef3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "39d31d4b04c14099bf89458586ff793c" member_type: VOTER last_known_addr { host: "127.19.194.1" port: 37593 } } peers { permanent_uuid: "b49bc56953ac498582361f7e613cec38" member_type: VOTER last_known_addr { host: "127.19.194.2" port: 38833 } } peers { permanent_uuid: "c5fb6bfd8ec4401fb0f9b3cf91ae5ef3" member_type: VOTER last_known_addr { host: "127.19.194.3" port: 43573 } }
I20260806 18:34:17.466243 22654 raft_consensus.cc:374] T 5b6f5f19d21e48c68ed4bb9d04975f23 P c5fb6bfd8ec4401fb0f9b3cf91ae5ef3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 18:34:17.466284 22654 raft_consensus.cc:729] T 5b6f5f19d21e48c68ed4bb9d04975f23 P c5fb6bfd8ec4401fb0f9b3cf91ae5ef3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c5fb6bfd8ec4401fb0f9b3cf91ae5ef3, State: Initialized, Role: FOLLOWER
I20260806 18:34:17.466413 22654 consensus_queue.cc:260] T 5b6f5f19d21e48c68ed4bb9d04975f23 P c5fb6bfd8ec4401fb0f9b3cf91ae5ef3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "39d31d4b04c14099bf89458586ff793c" member_type: VOTER last_known_addr { host: "127.19.194.1" port: 37593 } } peers { permanent_uuid: "b49bc56953ac498582361f7e613cec38" member_type: VOTER last_known_addr { host: "127.19.194.2" port: 38833 } } peers { permanent_uuid: "c5fb6bfd8ec4401fb0f9b3cf91ae5ef3" member_type: VOTER last_known_addr { host: "127.19.194.3" port: 43573 } }
I20260806 18:34:17.466843 22654 ts_tablet_manager.cc:1428] T 5b6f5f19d21e48c68ed4bb9d04975f23 P c5fb6bfd8ec4401fb0f9b3cf91ae5ef3: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260806 18:34:17.628965 22658 raft_consensus.cc:482] T 5b6f5f19d21e48c68ed4bb9d04975f23 P c5fb6bfd8ec4401fb0f9b3cf91ae5ef3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 18:34:17.629133 22658 raft_consensus.cc:504] T 5b6f5f19d21e48c68ed4bb9d04975f23 P c5fb6bfd8ec4401fb0f9b3cf91ae5ef3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "39d31d4b04c14099bf89458586ff793c" member_type: VOTER last_known_addr { host: "127.19.194.1" port: 37593 } } peers { permanent_uuid: "b49bc56953ac498582361f7e613cec38" member_type: VOTER last_known_addr { host: "127.19.194.2" port: 38833 } } peers { permanent_uuid: "c5fb6bfd8ec4401fb0f9b3cf91ae5ef3" member_type: VOTER last_known_addr { host: "127.19.194.3" port: 43573 } }
I20260806 18:34:17.630450 22658 leader_election.cc:288] T 5b6f5f19d21e48c68ed4bb9d04975f23 P c5fb6bfd8ec4401fb0f9b3cf91ae5ef3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 39d31d4b04c14099bf89458586ff793c (127.19.194.1:37593), b49bc56953ac498582361f7e613cec38 (127.19.194.2:38833)
W20260806 18:34:17.631193 22249 tablet.cc:2369] T 5b6f5f19d21e48c68ed4bb9d04975f23 P 39d31d4b04c14099bf89458586ff793c: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 18:34:17.634634 22660 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:34:17.630792 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.194.3:58659 (local address 127.19.194.1:37593)
0806 18:34:17.631081 (+ 289us) server_negotiation.cc:206] Beginning negotiation
0806 18:34:17.631094 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0806 18:34:17.631258 (+ 164us) server_negotiation.cc:407] Connection header received
0806 18:34:17.631393 (+ 135us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:34:17.631397 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:34:17.631450 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:34:17.631550 (+ 100us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 18:34:17.631999 (+ 449us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:34:17.632545 (+ 546us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:34:17.633630 (+ 1085us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:34:17.634055 (+ 425us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:34:17.634185 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:34:17.634240 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0806 18:34:17.634430 (+ 190us) server_negotiation.cc:299] Negotiation successful
0806 18:34:17.634505 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":205,"thread_start_us":91,"threads_started":1}
I20260806 18:34:17.634647 22659 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:34:17.630683 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.194.1:37593 (local address 127.19.194.3:58659)
0806 18:34:17.631072 (+ 389us) negotiation.cc:107] Waiting for socket to connect
0806 18:34:17.631090 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0806 18:34:17.631241 (+ 151us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:34:17.631685 (+ 444us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:34:17.631689 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:34:17.631712 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 18:34:17.631874 (+ 162us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:34:17.631887 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:34:17.632665 (+ 778us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:34:17.632669 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:34:17.633499 (+ 830us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:34:17.633508 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:34:17.634181 (+ 673us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:34:17.634185 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:34:17.634305 (+ 120us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:34:17.634325 (+ 20us) client_negotiation.cc:769] Sending connection context
0806 18:34:17.634410 (+ 85us) client_negotiation.cc:240] Negotiation successful
0806 18:34:17.634483 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":288,"mutex_wait_us":3,"thread_start_us":145,"threads_started":1}
I20260806 18:34:17.634708 22662 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:34:17.630853 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.194.3:53269 (local address 127.19.194.2:38833)
0806 18:34:17.631210 (+ 357us) server_negotiation.cc:206] Beginning negotiation
0806 18:34:17.631220 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0806 18:34:17.631513 (+ 293us) server_negotiation.cc:407] Connection header received
0806 18:34:17.631628 (+ 115us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:34:17.631632 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:34:17.631685 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:34:17.631772 (+ 87us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 18:34:17.632064 (+ 292us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:34:17.632595 (+ 531us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:34:17.633699 (+ 1104us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:34:17.634132 (+ 433us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:34:17.634260 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:34:17.634319 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0806 18:34:17.634515 (+ 196us) server_negotiation.cc:299] Negotiation successful
0806 18:34:17.634589 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":267,"thread_start_us":120,"threads_started":1}
I20260806 18:34:17.634729 22663 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:34:17.630683 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.194.2:38833 (local address 127.19.194.3:53269)
0806 18:34:17.631399 (+ 716us) negotiation.cc:107] Waiting for socket to connect
0806 18:34:17.631414 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0806 18:34:17.631529 (+ 115us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:34:17.631788 (+ 259us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:34:17.631790 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:34:17.631805 (+ 15us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 18:34:17.631940 (+ 135us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:34:17.631945 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:34:17.632721 (+ 776us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:34:17.632725 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:34:17.633580 (+ 855us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:34:17.633590 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:34:17.634252 (+ 662us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:34:17.634255 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:34:17.634391 (+ 136us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:34:17.634409 (+ 18us) client_negotiation.cc:769] Sending connection context
0806 18:34:17.634495 (+ 86us) client_negotiation.cc:240] Negotiation successful
0806 18:34:17.634564 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":649,"mutex_wait_us":446,"thread_start_us":108,"threads_started":1}
I20260806 18:34:17.635200 22344 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5b6f5f19d21e48c68ed4bb9d04975f23" candidate_uuid: "c5fb6bfd8ec4401fb0f9b3cf91ae5ef3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b49bc56953ac498582361f7e613cec38" is_pre_election: true
I20260806 18:34:17.635241 22202 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5b6f5f19d21e48c68ed4bb9d04975f23" candidate_uuid: "c5fb6bfd8ec4401fb0f9b3cf91ae5ef3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "39d31d4b04c14099bf89458586ff793c" is_pre_election: true
I20260806 18:34:17.635362 22344 raft_consensus.cc:2454] T 5b6f5f19d21e48c68ed4bb9d04975f23 P b49bc56953ac498582361f7e613cec38 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c5fb6bfd8ec4401fb0f9b3cf91ae5ef3 in term 0.
I20260806 18:34:17.635362 22202 raft_consensus.cc:2454] T 5b6f5f19d21e48c68ed4bb9d04975f23 P 39d31d4b04c14099bf89458586ff793c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c5fb6bfd8ec4401fb0f9b3cf91ae5ef3 in term 0.
I20260806 18:34:17.635715 22419 leader_election.cc:302] T 5b6f5f19d21e48c68ed4bb9d04975f23 P c5fb6bfd8ec4401fb0f9b3cf91ae5ef3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b49bc56953ac498582361f7e613cec38, c5fb6bfd8ec4401fb0f9b3cf91ae5ef3; no voters:
I20260806 18:34:17.635967 22658 raft_consensus.cc:2781] T 5b6f5f19d21e48c68ed4bb9d04975f23 P c5fb6bfd8ec4401fb0f9b3cf91ae5ef3 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 18:34:17.636041 22658 raft_consensus.cc:482] T 5b6f5f19d21e48c68ed4bb9d04975f23 P c5fb6bfd8ec4401fb0f9b3cf91ae5ef3 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 18:34:17.636097 22658 raft_consensus.cc:3037] T 5b6f5f19d21e48c68ed4bb9d04975f23 P c5fb6bfd8ec4401fb0f9b3cf91ae5ef3 [term 0 FOLLOWER]: Advancing to term 1
I20260806 18:34:17.637099 22658 raft_consensus.cc:504] T 5b6f5f19d21e48c68ed4bb9d04975f23 P c5fb6bfd8ec4401fb0f9b3cf91ae5ef3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "39d31d4b04c14099bf89458586ff793c" member_type: VOTER last_known_addr { host: "127.19.194.1" port: 37593 } } peers { permanent_uuid: "b49bc56953ac498582361f7e613cec38" member_type: VOTER last_known_addr { host: "127.19.194.2" port: 38833 } } peers { permanent_uuid: "c5fb6bfd8ec4401fb0f9b3cf91ae5ef3" member_type: VOTER last_known_addr { host: "127.19.194.3" port: 43573 } }
I20260806 18:34:17.637449 22658 leader_election.cc:288] T 5b6f5f19d21e48c68ed4bb9d04975f23 P c5fb6bfd8ec4401fb0f9b3cf91ae5ef3 [CANDIDATE]: Term 1 election: Requested vote from peers 39d31d4b04c14099bf89458586ff793c (127.19.194.1:37593), b49bc56953ac498582361f7e613cec38 (127.19.194.2:38833)
I20260806 18:34:17.637781 22202 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5b6f5f19d21e48c68ed4bb9d04975f23" candidate_uuid: "c5fb6bfd8ec4401fb0f9b3cf91ae5ef3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "39d31d4b04c14099bf89458586ff793c"
I20260806 18:34:17.637800 22344 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5b6f5f19d21e48c68ed4bb9d04975f23" candidate_uuid: "c5fb6bfd8ec4401fb0f9b3cf91ae5ef3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b49bc56953ac498582361f7e613cec38"
I20260806 18:34:17.637935 22202 raft_consensus.cc:3037] T 5b6f5f19d21e48c68ed4bb9d04975f23 P 39d31d4b04c14099bf89458586ff793c [term 0 FOLLOWER]: Advancing to term 1
I20260806 18:34:17.637919 22344 raft_consensus.cc:3037] T 5b6f5f19d21e48c68ed4bb9d04975f23 P b49bc56953ac498582361f7e613cec38 [term 0 FOLLOWER]: Advancing to term 1
I20260806 18:34:17.638931 22202 raft_consensus.cc:2454] T 5b6f5f19d21e48c68ed4bb9d04975f23 P 39d31d4b04c14099bf89458586ff793c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c5fb6bfd8ec4401fb0f9b3cf91ae5ef3 in term 1.
I20260806 18:34:17.638953 22344 raft_consensus.cc:2454] T 5b6f5f19d21e48c68ed4bb9d04975f23 P b49bc56953ac498582361f7e613cec38 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c5fb6bfd8ec4401fb0f9b3cf91ae5ef3 in term 1.
I20260806 18:34:17.639264 22418 leader_election.cc:302] T 5b6f5f19d21e48c68ed4bb9d04975f23 P c5fb6bfd8ec4401fb0f9b3cf91ae5ef3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 39d31d4b04c14099bf89458586ff793c, c5fb6bfd8ec4401fb0f9b3cf91ae5ef3; no voters:
I20260806 18:34:17.639446 22658 raft_consensus.cc:2781] T 5b6f5f19d21e48c68ed4bb9d04975f23 P c5fb6bfd8ec4401fb0f9b3cf91ae5ef3 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 18:34:17.639655 22658 raft_consensus.cc:686] T 5b6f5f19d21e48c68ed4bb9d04975f23 P c5fb6bfd8ec4401fb0f9b3cf91ae5ef3 [term 1 LEADER]: Becoming Leader. State: Replica: c5fb6bfd8ec4401fb0f9b3cf91ae5ef3, State: Running, Role: LEADER
I20260806 18:34:17.639904 22658 consensus_queue.cc:237] T 5b6f5f19d21e48c68ed4bb9d04975f23 P c5fb6bfd8ec4401fb0f9b3cf91ae5ef3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "39d31d4b04c14099bf89458586ff793c" member_type: VOTER last_known_addr { host: "127.19.194.1" port: 37593 } } peers { permanent_uuid: "b49bc56953ac498582361f7e613cec38" member_type: VOTER last_known_addr { host: "127.19.194.2" port: 38833 } } peers { permanent_uuid: "c5fb6bfd8ec4401fb0f9b3cf91ae5ef3" member_type: VOTER last_known_addr { host: "127.19.194.3" port: 43573 } }
I20260806 18:34:17.642145 22037 catalog_manager.cc:5495] T 5b6f5f19d21e48c68ed4bb9d04975f23 P c5fb6bfd8ec4401fb0f9b3cf91ae5ef3 reported cstate change: term changed from 0 to 1, leader changed from <none> to c5fb6bfd8ec4401fb0f9b3cf91ae5ef3 (127.19.194.3). New cstate: current_term: 1 leader_uuid: "c5fb6bfd8ec4401fb0f9b3cf91ae5ef3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "39d31d4b04c14099bf89458586ff793c" member_type: VOTER last_known_addr { host: "127.19.194.1" port: 37593 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b49bc56953ac498582361f7e613cec38" member_type: VOTER last_known_addr { host: "127.19.194.2" port: 38833 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c5fb6bfd8ec4401fb0f9b3cf91ae5ef3" member_type: VOTER last_known_addr { host: "127.19.194.3" port: 43573 } health_report { overall_health: HEALTHY } } }
I20260806 18:34:17.657457 22667 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:34:17.654113 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36894 (local address 127.19.194.3:43573)
0806 18:34:17.654336 (+ 223us) server_negotiation.cc:206] Beginning negotiation
0806 18:34:17.654343 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 18:34:17.654354 (+ 11us) server_negotiation.cc:407] Connection header received
0806 18:34:17.654414 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:34:17.654416 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:34:17.654464 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:34:17.654561 (+ 97us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 18:34:17.654777 (+ 216us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:34:17.655340 (+ 563us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:34:17.656132 (+ 792us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:34:17.656365 (+ 233us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:34:17.656439 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:34:17.656662 (+ 223us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 18:34:17.657087 (+ 425us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 18:34:17.657172 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0806 18:34:17.657235 (+ 63us) server_negotiation.cc:299] Negotiation successful
0806 18:34:17.657278 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":160,"thread_start_us":94,"threads_started":1}
I20260806 18:34:17.666800 22662 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:34:17.664155 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.194.3:35457 (local address 127.19.194.2:38833)
0806 18:34:17.664261 (+ 106us) server_negotiation.cc:206] Beginning negotiation
0806 18:34:17.664264 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 18:34:17.664290 (+ 26us) server_negotiation.cc:407] Connection header received
0806 18:34:17.664403 (+ 113us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:34:17.664406 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:34:17.664445 (+ 39us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:34:17.664512 (+ 67us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 18:34:17.664708 (+ 196us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:34:17.665161 (+ 453us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:34:17.665962 (+ 801us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:34:17.666159 (+ 197us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:34:17.666231 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:34:17.666480 (+ 249us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 18:34:17.666540 (+ 60us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 18:34:17.666591 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0806 18:34:17.666665 (+ 74us) server_negotiation.cc:299] Negotiation successful
0806 18:34:17.666704 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260806 18:34:17.666836 22670 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:34:17.663973 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.194.2:38833 (local address 127.19.194.3:35457)
0806 18:34:17.664231 (+ 258us) negotiation.cc:107] Waiting for socket to connect
0806 18:34:17.664243 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0806 18:34:17.664326 (+ 83us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:34:17.664533 (+ 207us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:34:17.664535 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:34:17.664541 (+ 6us) client_negotiation.cc:189] Negotiated authn=TOKEN
0806 18:34:17.664627 (+ 86us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:34:17.664630 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:34:17.665252 (+ 622us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:34:17.665254 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:34:17.665866 (+ 612us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:34:17.665874 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:34:17.666264 (+ 390us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:34:17.666266 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:34:17.666330 (+ 64us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:34:17.666370 (+ 40us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0806 18:34:17.666608 (+ 238us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0806 18:34:17.666612 (+ 4us) client_negotiation.cc:769] Sending connection context
0806 18:34:17.666661 (+ 49us) client_negotiation.cc:240] Negotiation successful
0806 18:34:17.666724 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":169,"thread_start_us":94,"threads_started":1}
I20260806 18:34:17.672499 22202 raft_consensus.cc:1261] T 5b6f5f19d21e48c68ed4bb9d04975f23 P 39d31d4b04c14099bf89458586ff793c [term 1 FOLLOWER]: Refusing update from remote peer c5fb6bfd8ec4401fb0f9b3cf91ae5ef3: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 18:34:17.672562 22344 raft_consensus.cc:1261] T 5b6f5f19d21e48c68ed4bb9d04975f23 P b49bc56953ac498582361f7e613cec38 [term 1 FOLLOWER]: Refusing update from remote peer c5fb6bfd8ec4401fb0f9b3cf91ae5ef3: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 18:34:17.672942 22664 consensus_queue.cc:1035] T 5b6f5f19d21e48c68ed4bb9d04975f23 P c5fb6bfd8ec4401fb0f9b3cf91ae5ef3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "39d31d4b04c14099bf89458586ff793c" member_type: VOTER last_known_addr { host: "127.19.194.1" port: 37593 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 18:34:17.673102 22658 consensus_queue.cc:1035] T 5b6f5f19d21e48c68ed4bb9d04975f23 P c5fb6bfd8ec4401fb0f9b3cf91ae5ef3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b49bc56953ac498582361f7e613cec38" member_type: VOTER last_known_addr { host: "127.19.194.2" port: 38833 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 18:34:17.690187 22667 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:34:17.686543 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.194.2:50121 (local address 127.19.194.3:43573)
0806 18:34:17.686748 (+ 205us) server_negotiation.cc:206] Beginning negotiation
0806 18:34:17.686752 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 18:34:17.686795 (+ 43us) server_negotiation.cc:407] Connection header received
0806 18:34:17.686976 (+ 181us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:34:17.686980 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:34:17.687032 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:34:17.687106 (+ 74us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 18:34:17.687377 (+ 271us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:34:17.687882 (+ 505us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:34:17.688921 (+ 1039us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:34:17.689268 (+ 347us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:34:17.689402 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:34:17.689679 (+ 277us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 18:34:17.689760 (+ 81us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 18:34:17.689831 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0806 18:34:17.690006 (+ 175us) server_negotiation.cc:299] Negotiation successful
0806 18:34:17.690070 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":138}
I20260806 18:34:17.690403 22675 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:34:17.686419 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.194.3:43573 (local address 127.19.194.2:50121)
0806 18:34:17.686722 (+ 303us) negotiation.cc:107] Waiting for socket to connect
0806 18:34:17.686736 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0806 18:34:17.686837 (+ 101us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:34:17.687156 (+ 319us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:34:17.687159 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:34:17.687168 (+ 9us) client_negotiation.cc:189] Negotiated authn=TOKEN
0806 18:34:17.687269 (+ 101us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:34:17.687274 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:34:17.687990 (+ 716us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:34:17.687995 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:34:17.688757 (+ 762us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:34:17.688770 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:34:17.689416 (+ 646us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:34:17.689420 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:34:17.689524 (+ 104us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:34:17.689552 (+ 28us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0806 18:34:17.689890 (+ 338us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0806 18:34:17.689899 (+ 9us) client_negotiation.cc:769] Sending connection context
0806 18:34:17.689976 (+ 77us) client_negotiation.cc:240] Negotiation successful
0806 18:34:17.690097 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":221,"thread_start_us":97,"threads_started":1}
I20260806 18:34:17.712827 22040 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:40694:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 18:34:17.713023 22040 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:40694:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 18:34:17.715090 22040 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 805a3be867fa43bdafe4c8974a803d51: Sending DeleteTablet for 3 replicas of tablet 5b6f5f19d21e48c68ed4bb9d04975f23
I20260806 18:34:17.715597 22180 tablet_service.cc:1514] Processing DeleteTablet for tablet 5b6f5f19d21e48c68ed4bb9d04975f23 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 18:34:17 UTC) from {username='kudu', principal='kudu/127.19.194.62@KRBTEST.COM'} at 127.0.0.1:49642
I20260806 18:34:17.715770 22641 tablet_replica.cc:330] T 5b6f5f19d21e48c68ed4bb9d04975f23 P 39d31d4b04c14099bf89458586ff793c: stopping tablet replica
I20260806 18:34:17.715883 22641 raft_consensus.cc:2229] T 5b6f5f19d21e48c68ed4bb9d04975f23 P 39d31d4b04c14099bf89458586ff793c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 18:34:17.715903 22324 tablet_service.cc:1514] Processing DeleteTablet for tablet 5b6f5f19d21e48c68ed4bb9d04975f23 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 18:34:17 UTC) from {username='kudu', principal='kudu/127.19.194.62@KRBTEST.COM'} at 127.0.0.1:55086
I20260806 18:34:17.716001 22641 raft_consensus.cc:2258] T 5b6f5f19d21e48c68ed4bb9d04975f23 P 39d31d4b04c14099bf89458586ff793c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 18:34:17.716187 22643 tablet_replica.cc:330] T 5b6f5f19d21e48c68ed4bb9d04975f23 P b49bc56953ac498582361f7e613cec38: stopping tablet replica
I20260806 18:34:17.716356 22643 raft_consensus.cc:2229] T 5b6f5f19d21e48c68ed4bb9d04975f23 P b49bc56953ac498582361f7e613cec38 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 18:34:17.716346 22040 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:40694:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260806 18:34:17.716490 22643 raft_consensus.cc:2258] T 5b6f5f19d21e48c68ed4bb9d04975f23 P b49bc56953ac498582361f7e613cec38 [term 1 FOLLOWER]: Raft consensus is shut down!
W20260806 18:34:17.716764 22040 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 18:34:17.717149 22468 tablet_service.cc:1514] Processing DeleteTablet for tablet 5b6f5f19d21e48c68ed4bb9d04975f23 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 18:34:17 UTC) from {username='kudu', principal='kudu/127.19.194.62@KRBTEST.COM'} at 127.0.0.1:36884
I20260806 18:34:17.717307 22642 tablet_replica.cc:330] T 5b6f5f19d21e48c68ed4bb9d04975f23 P c5fb6bfd8ec4401fb0f9b3cf91ae5ef3: stopping tablet replica
I20260806 18:34:17.717415 22642 raft_consensus.cc:2229] T 5b6f5f19d21e48c68ed4bb9d04975f23 P c5fb6bfd8ec4401fb0f9b3cf91ae5ef3 [term 1 LEADER]: Raft consensus shutting down.
I20260806 18:34:17.717669 22642 raft_consensus.cc:2258] T 5b6f5f19d21e48c68ed4bb9d04975f23 P c5fb6bfd8ec4401fb0f9b3cf91ae5ef3 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 18:34:17.717684 22641 ts_tablet_manager.cc:1905] T 5b6f5f19d21e48c68ed4bb9d04975f23 P 39d31d4b04c14099bf89458586ff793c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 18:34:17.717991 22643 ts_tablet_manager.cc:1905] T 5b6f5f19d21e48c68ed4bb9d04975f23 P b49bc56953ac498582361f7e613cec38: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 18:34:17.720105 22642 ts_tablet_manager.cc:1905] T 5b6f5f19d21e48c68ed4bb9d04975f23 P c5fb6bfd8ec4401fb0f9b3cf91ae5ef3: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 18:34:17.720438 22641 ts_tablet_manager.cc:1918] T 5b6f5f19d21e48c68ed4bb9d04975f23 P 39d31d4b04c14099bf89458586ff793c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260806 18:34:17.720503 22641 log.cc:1196] T 5b6f5f19d21e48c68ed4bb9d04975f23 P 39d31d4b04c14099bf89458586ff793c: Deleting WAL directory at /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786041246094603-20232-0/minicluster-data/ts-0/wal/wals/5b6f5f19d21e48c68ed4bb9d04975f23
I20260806 18:34:17.720610 22643 ts_tablet_manager.cc:1918] T 5b6f5f19d21e48c68ed4bb9d04975f23 P b49bc56953ac498582361f7e613cec38: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260806 18:34:17.720679 22643 log.cc:1196] T 5b6f5f19d21e48c68ed4bb9d04975f23 P b49bc56953ac498582361f7e613cec38: Deleting WAL directory at /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786041246094603-20232-0/minicluster-data/ts-1/wal/wals/5b6f5f19d21e48c68ed4bb9d04975f23
I20260806 18:34:17.720747 22641 ts_tablet_manager.cc:1939] T 5b6f5f19d21e48c68ed4bb9d04975f23 P 39d31d4b04c14099bf89458586ff793c: Deleting consensus metadata
I20260806 18:34:17.720935 22643 ts_tablet_manager.cc:1939] T 5b6f5f19d21e48c68ed4bb9d04975f23 P b49bc56953ac498582361f7e613cec38: Deleting consensus metadata
I20260806 18:34:17.721633 22025 catalog_manager.cc:4841] TS b49bc56953ac498582361f7e613cec38 (127.19.194.2:38833): tablet 5b6f5f19d21e48c68ed4bb9d04975f23 (table test-table [id=1ceddd20b4b143528573da4291425588]) successfully deleted
I20260806 18:34:17.722039 22024 catalog_manager.cc:4841] TS 39d31d4b04c14099bf89458586ff793c (127.19.194.1:37593): tablet 5b6f5f19d21e48c68ed4bb9d04975f23 (table test-table [id=1ceddd20b4b143528573da4291425588]) successfully deleted
I20260806 18:34:17.722666 22642 ts_tablet_manager.cc:1918] T 5b6f5f19d21e48c68ed4bb9d04975f23 P c5fb6bfd8ec4401fb0f9b3cf91ae5ef3: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260806 18:34:17.722740 22642 log.cc:1196] T 5b6f5f19d21e48c68ed4bb9d04975f23 P c5fb6bfd8ec4401fb0f9b3cf91ae5ef3: Deleting WAL directory at /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786041246094603-20232-0/minicluster-data/ts-2/wal/wals/5b6f5f19d21e48c68ed4bb9d04975f23
I20260806 18:34:17.723008 22642 ts_tablet_manager.cc:1939] T 5b6f5f19d21e48c68ed4bb9d04975f23 P c5fb6bfd8ec4401fb0f9b3cf91ae5ef3: Deleting consensus metadata
I20260806 18:34:17.723963 22024 catalog_manager.cc:4841] TS c5fb6bfd8ec4401fb0f9b3cf91ae5ef3 (127.19.194.3:43573): tablet 5b6f5f19d21e48c68ed4bb9d04975f23 (table test-table [id=1ceddd20b4b143528573da4291425588]) successfully deleted
I20260806 18:34:17.724203 22324 tablet_service.cc:1467] Processing CreateTablet for tablet 19d56f4fd4f64470b6b0201f5586cb54 (DEFAULT_TABLE table=test-table [id=d0aa5da933f140cea14c090e10ff6fca]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 18:34:17.724478 22324 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 19d56f4fd4f64470b6b0201f5586cb54. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 18:34:17.724442 22180 tablet_service.cc:1467] Processing CreateTablet for tablet 19d56f4fd4f64470b6b0201f5586cb54 (DEFAULT_TABLE table=test-table [id=d0aa5da933f140cea14c090e10ff6fca]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 18:34:17.724534 22468 tablet_service.cc:1467] Processing CreateTablet for tablet 19d56f4fd4f64470b6b0201f5586cb54 (DEFAULT_TABLE table=test-table [id=d0aa5da933f140cea14c090e10ff6fca]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 18:34:17.724653 22180 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 19d56f4fd4f64470b6b0201f5586cb54. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 18:34:17.724763 22468 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 19d56f4fd4f64470b6b0201f5586cb54. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 18:34:17.726836 22653 tablet_bootstrap.cc:492] T 19d56f4fd4f64470b6b0201f5586cb54 P 39d31d4b04c14099bf89458586ff793c: Bootstrap starting.
I20260806 18:34:17.726917 22652 tablet_bootstrap.cc:492] T 19d56f4fd4f64470b6b0201f5586cb54 P b49bc56953ac498582361f7e613cec38: Bootstrap starting.
I20260806 18:34:17.727648 22652 tablet_bootstrap.cc:654] T 19d56f4fd4f64470b6b0201f5586cb54 P b49bc56953ac498582361f7e613cec38: Neither blocks nor log segments found. Creating new log.
I20260806 18:34:17.727685 22653 tablet_bootstrap.cc:654] T 19d56f4fd4f64470b6b0201f5586cb54 P 39d31d4b04c14099bf89458586ff793c: Neither blocks nor log segments found. Creating new log.
I20260806 18:34:17.728211 22654 tablet_bootstrap.cc:492] T 19d56f4fd4f64470b6b0201f5586cb54 P c5fb6bfd8ec4401fb0f9b3cf91ae5ef3: Bootstrap starting.
I20260806 18:34:17.728796 22652 tablet_bootstrap.cc:492] T 19d56f4fd4f64470b6b0201f5586cb54 P b49bc56953ac498582361f7e613cec38: No bootstrap required, opened a new log
I20260806 18:34:17.728924 22652 ts_tablet_manager.cc:1397] T 19d56f4fd4f64470b6b0201f5586cb54 P b49bc56953ac498582361f7e613cec38: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260806 18:34:17.729157 22654 tablet_bootstrap.cc:654] T 19d56f4fd4f64470b6b0201f5586cb54 P c5fb6bfd8ec4401fb0f9b3cf91ae5ef3: Neither blocks nor log segments found. Creating new log.
I20260806 18:34:17.729408 22652 raft_consensus.cc:348] T 19d56f4fd4f64470b6b0201f5586cb54 P b49bc56953ac498582361f7e613cec38 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "39d31d4b04c14099bf89458586ff793c" member_type: VOTER last_known_addr { host: "127.19.194.1" port: 37593 } } peers { permanent_uuid: "b49bc56953ac498582361f7e613cec38" member_type: VOTER last_known_addr { host: "127.19.194.2" port: 38833 } } peers { permanent_uuid: "c5fb6bfd8ec4401fb0f9b3cf91ae5ef3" member_type: VOTER last_known_addr { host: "127.19.194.3" port: 43573 } }
I20260806 18:34:17.729526 22652 raft_consensus.cc:374] T 19d56f4fd4f64470b6b0201f5586cb54 P b49bc56953ac498582361f7e613cec38 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 18:34:17.729563 22652 raft_consensus.cc:729] T 19d56f4fd4f64470b6b0201f5586cb54 P b49bc56953ac498582361f7e613cec38 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b49bc56953ac498582361f7e613cec38, State: Initialized, Role: FOLLOWER
I20260806 18:34:17.729686 22652 consensus_queue.cc:260] T 19d56f4fd4f64470b6b0201f5586cb54 P b49bc56953ac498582361f7e613cec38 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "39d31d4b04c14099bf89458586ff793c" member_type: VOTER last_known_addr { host: "127.19.194.1" port: 37593 } } peers { permanent_uuid: "b49bc56953ac498582361f7e613cec38" member_type: VOTER last_known_addr { host: "127.19.194.2" port: 38833 } } peers { permanent_uuid: "c5fb6bfd8ec4401fb0f9b3cf91ae5ef3" member_type: VOTER last_known_addr { host: "127.19.194.3" port: 43573 } }
I20260806 18:34:17.729977 22652 ts_tablet_manager.cc:1428] T 19d56f4fd4f64470b6b0201f5586cb54 P b49bc56953ac498582361f7e613cec38: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260806 18:34:17.730144 22653 tablet_bootstrap.cc:492] T 19d56f4fd4f64470b6b0201f5586cb54 P 39d31d4b04c14099bf89458586ff793c: No bootstrap required, opened a new log
I20260806 18:34:17.730237 22653 ts_tablet_manager.cc:1397] T 19d56f4fd4f64470b6b0201f5586cb54 P 39d31d4b04c14099bf89458586ff793c: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260806 18:34:17.730484 22654 tablet_bootstrap.cc:492] T 19d56f4fd4f64470b6b0201f5586cb54 P c5fb6bfd8ec4401fb0f9b3cf91ae5ef3: No bootstrap required, opened a new log
I20260806 18:34:17.730556 22654 ts_tablet_manager.cc:1397] T 19d56f4fd4f64470b6b0201f5586cb54 P c5fb6bfd8ec4401fb0f9b3cf91ae5ef3: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260806 18:34:17.730672 22653 raft_consensus.cc:348] T 19d56f4fd4f64470b6b0201f5586cb54 P 39d31d4b04c14099bf89458586ff793c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "39d31d4b04c14099bf89458586ff793c" member_type: VOTER last_known_addr { host: "127.19.194.1" port: 37593 } } peers { permanent_uuid: "b49bc56953ac498582361f7e613cec38" member_type: VOTER last_known_addr { host: "127.19.194.2" port: 38833 } } peers { permanent_uuid: "c5fb6bfd8ec4401fb0f9b3cf91ae5ef3" member_type: VOTER last_known_addr { host: "127.19.194.3" port: 43573 } }
I20260806 18:34:17.730770 22653 raft_consensus.cc:374] T 19d56f4fd4f64470b6b0201f5586cb54 P 39d31d4b04c14099bf89458586ff793c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 18:34:17.730803 22653 raft_consensus.cc:729] T 19d56f4fd4f64470b6b0201f5586cb54 P 39d31d4b04c14099bf89458586ff793c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 39d31d4b04c14099bf89458586ff793c, State: Initialized, Role: FOLLOWER
I20260806 18:34:17.730911 22653 consensus_queue.cc:260] T 19d56f4fd4f64470b6b0201f5586cb54 P 39d31d4b04c14099bf89458586ff793c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "39d31d4b04c14099bf89458586ff793c" member_type: VOTER last_known_addr { host: "127.19.194.1" port: 37593 } } peers { permanent_uuid: "b49bc56953ac498582361f7e613cec38" member_type: VOTER last_known_addr { host: "127.19.194.2" port: 38833 } } peers { permanent_uuid: "c5fb6bfd8ec4401fb0f9b3cf91ae5ef3" member_type: VOTER last_known_addr { host: "127.19.194.3" port: 43573 } }
I20260806 18:34:17.730935 22654 raft_consensus.cc:348] T 19d56f4fd4f64470b6b0201f5586cb54 P c5fb6bfd8ec4401fb0f9b3cf91ae5ef3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "39d31d4b04c14099bf89458586ff793c" member_type: VOTER last_known_addr { host: "127.19.194.1" port: 37593 } } peers { permanent_uuid: "b49bc56953ac498582361f7e613cec38" member_type: VOTER last_known_addr { host: "127.19.194.2" port: 38833 } } peers { permanent_uuid: "c5fb6bfd8ec4401fb0f9b3cf91ae5ef3" member_type: VOTER last_known_addr { host: "127.19.194.3" port: 43573 } }
I20260806 18:34:17.731033 22654 raft_consensus.cc:374] T 19d56f4fd4f64470b6b0201f5586cb54 P c5fb6bfd8ec4401fb0f9b3cf91ae5ef3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 18:34:17.731076 22654 raft_consensus.cc:729] T 19d56f4fd4f64470b6b0201f5586cb54 P c5fb6bfd8ec4401fb0f9b3cf91ae5ef3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c5fb6bfd8ec4401fb0f9b3cf91ae5ef3, State: Initialized, Role: FOLLOWER
I20260806 18:34:17.731199 22654 consensus_queue.cc:260] T 19d56f4fd4f64470b6b0201f5586cb54 P c5fb6bfd8ec4401fb0f9b3cf91ae5ef3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "39d31d4b04c14099bf89458586ff793c" member_type: VOTER last_known_addr { host: "127.19.194.1" port: 37593 } } peers { permanent_uuid: "b49bc56953ac498582361f7e613cec38" member_type: VOTER last_known_addr { host: "127.19.194.2" port: 38833 } } peers { permanent_uuid: "c5fb6bfd8ec4401fb0f9b3cf91ae5ef3" member_type: VOTER last_known_addr { host: "127.19.194.3" port: 43573 } }
I20260806 18:34:17.731241 22653 ts_tablet_manager.cc:1428] T 19d56f4fd4f64470b6b0201f5586cb54 P 39d31d4b04c14099bf89458586ff793c: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260806 18:34:17.731441 22654 ts_tablet_manager.cc:1428] T 19d56f4fd4f64470b6b0201f5586cb54 P c5fb6bfd8ec4401fb0f9b3cf91ae5ef3: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260806 18:34:17.804692 22627 raft_consensus.cc:482] T 19d56f4fd4f64470b6b0201f5586cb54 P b49bc56953ac498582361f7e613cec38 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 18:34:17.804842 22627 raft_consensus.cc:504] T 19d56f4fd4f64470b6b0201f5586cb54 P b49bc56953ac498582361f7e613cec38 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "39d31d4b04c14099bf89458586ff793c" member_type: VOTER last_known_addr { host: "127.19.194.1" port: 37593 } } peers { permanent_uuid: "b49bc56953ac498582361f7e613cec38" member_type: VOTER last_known_addr { host: "127.19.194.2" port: 38833 } } peers { permanent_uuid: "c5fb6bfd8ec4401fb0f9b3cf91ae5ef3" member_type: VOTER last_known_addr { host: "127.19.194.3" port: 43573 } }
I20260806 18:34:17.805347 22627 leader_election.cc:288] T 19d56f4fd4f64470b6b0201f5586cb54 P b49bc56953ac498582361f7e613cec38 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 39d31d4b04c14099bf89458586ff793c (127.19.194.1:37593), c5fb6bfd8ec4401fb0f9b3cf91ae5ef3 (127.19.194.3:43573)
I20260806 18:34:17.805809 22488 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "19d56f4fd4f64470b6b0201f5586cb54" candidate_uuid: "b49bc56953ac498582361f7e613cec38" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c5fb6bfd8ec4401fb0f9b3cf91ae5ef3" is_pre_election: true
I20260806 18:34:17.805838 22202 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "19d56f4fd4f64470b6b0201f5586cb54" candidate_uuid: "b49bc56953ac498582361f7e613cec38" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "39d31d4b04c14099bf89458586ff793c" is_pre_election: true
I20260806 18:34:17.805970 22488 raft_consensus.cc:2454] T 19d56f4fd4f64470b6b0201f5586cb54 P c5fb6bfd8ec4401fb0f9b3cf91ae5ef3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b49bc56953ac498582361f7e613cec38 in term 0.
I20260806 18:34:17.806197 22202 raft_consensus.cc:2454] T 19d56f4fd4f64470b6b0201f5586cb54 P 39d31d4b04c14099bf89458586ff793c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b49bc56953ac498582361f7e613cec38 in term 0.
I20260806 18:34:17.806344 22274 leader_election.cc:302] T 19d56f4fd4f64470b6b0201f5586cb54 P b49bc56953ac498582361f7e613cec38 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b49bc56953ac498582361f7e613cec38, c5fb6bfd8ec4401fb0f9b3cf91ae5ef3; no voters:
I20260806 18:34:17.806684 22627 raft_consensus.cc:2781] T 19d56f4fd4f64470b6b0201f5586cb54 P b49bc56953ac498582361f7e613cec38 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 18:34:17.806751 22627 raft_consensus.cc:482] T 19d56f4fd4f64470b6b0201f5586cb54 P b49bc56953ac498582361f7e613cec38 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 18:34:17.806799 22627 raft_consensus.cc:3037] T 19d56f4fd4f64470b6b0201f5586cb54 P b49bc56953ac498582361f7e613cec38 [term 0 FOLLOWER]: Advancing to term 1
I20260806 18:34:17.807955 22627 raft_consensus.cc:504] T 19d56f4fd4f64470b6b0201f5586cb54 P b49bc56953ac498582361f7e613cec38 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "39d31d4b04c14099bf89458586ff793c" member_type: VOTER last_known_addr { host: "127.19.194.1" port: 37593 } } peers { permanent_uuid: "b49bc56953ac498582361f7e613cec38" member_type: VOTER last_known_addr { host: "127.19.194.2" port: 38833 } } peers { permanent_uuid: "c5fb6bfd8ec4401fb0f9b3cf91ae5ef3" member_type: VOTER last_known_addr { host: "127.19.194.3" port: 43573 } }
I20260806 18:34:17.808311 22627 leader_election.cc:288] T 19d56f4fd4f64470b6b0201f5586cb54 P b49bc56953ac498582361f7e613cec38 [CANDIDATE]: Term 1 election: Requested vote from peers 39d31d4b04c14099bf89458586ff793c (127.19.194.1:37593), c5fb6bfd8ec4401fb0f9b3cf91ae5ef3 (127.19.194.3:43573)
I20260806 18:34:17.808653 22202 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "19d56f4fd4f64470b6b0201f5586cb54" candidate_uuid: "b49bc56953ac498582361f7e613cec38" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "39d31d4b04c14099bf89458586ff793c"
I20260806 18:34:17.808777 22202 raft_consensus.cc:3037] T 19d56f4fd4f64470b6b0201f5586cb54 P 39d31d4b04c14099bf89458586ff793c [term 0 FOLLOWER]: Advancing to term 1
I20260806 18:34:17.808791 22488 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "19d56f4fd4f64470b6b0201f5586cb54" candidate_uuid: "b49bc56953ac498582361f7e613cec38" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c5fb6bfd8ec4401fb0f9b3cf91ae5ef3"
I20260806 18:34:17.808923 22488 raft_consensus.cc:3037] T 19d56f4fd4f64470b6b0201f5586cb54 P c5fb6bfd8ec4401fb0f9b3cf91ae5ef3 [term 0 FOLLOWER]: Advancing to term 1
I20260806 18:34:17.809839 22488 raft_consensus.cc:2454] T 19d56f4fd4f64470b6b0201f5586cb54 P c5fb6bfd8ec4401fb0f9b3cf91ae5ef3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b49bc56953ac498582361f7e613cec38 in term 1.
I20260806 18:34:17.809975 22202 raft_consensus.cc:2454] T 19d56f4fd4f64470b6b0201f5586cb54 P 39d31d4b04c14099bf89458586ff793c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b49bc56953ac498582361f7e613cec38 in term 1.
I20260806 18:34:17.810171 22274 leader_election.cc:302] T 19d56f4fd4f64470b6b0201f5586cb54 P b49bc56953ac498582361f7e613cec38 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b49bc56953ac498582361f7e613cec38, c5fb6bfd8ec4401fb0f9b3cf91ae5ef3; no voters:
I20260806 18:34:17.810326 22627 raft_consensus.cc:2781] T 19d56f4fd4f64470b6b0201f5586cb54 P b49bc56953ac498582361f7e613cec38 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 18:34:17.810500 22627 raft_consensus.cc:686] T 19d56f4fd4f64470b6b0201f5586cb54 P b49bc56953ac498582361f7e613cec38 [term 1 LEADER]: Becoming Leader. State: Replica: b49bc56953ac498582361f7e613cec38, State: Running, Role: LEADER
I20260806 18:34:17.810650 22627 consensus_queue.cc:237] T 19d56f4fd4f64470b6b0201f5586cb54 P b49bc56953ac498582361f7e613cec38 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "39d31d4b04c14099bf89458586ff793c" member_type: VOTER last_known_addr { host: "127.19.194.1" port: 37593 } } peers { permanent_uuid: "b49bc56953ac498582361f7e613cec38" member_type: VOTER last_known_addr { host: "127.19.194.2" port: 38833 } } peers { permanent_uuid: "c5fb6bfd8ec4401fb0f9b3cf91ae5ef3" member_type: VOTER last_known_addr { host: "127.19.194.3" port: 43573 } }
I20260806 18:34:17.812323 22038 catalog_manager.cc:5495] T 19d56f4fd4f64470b6b0201f5586cb54 P b49bc56953ac498582361f7e613cec38 reported cstate change: term changed from 0 to 1, leader changed from <none> to b49bc56953ac498582361f7e613cec38 (127.19.194.2). New cstate: current_term: 1 leader_uuid: "b49bc56953ac498582361f7e613cec38" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "39d31d4b04c14099bf89458586ff793c" member_type: VOTER last_known_addr { host: "127.19.194.1" port: 37593 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b49bc56953ac498582361f7e613cec38" member_type: VOTER last_known_addr { host: "127.19.194.2" port: 38833 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c5fb6bfd8ec4401fb0f9b3cf91ae5ef3" member_type: VOTER last_known_addr { host: "127.19.194.3" port: 43573 } health_report { overall_health: UNKNOWN } } }
I20260806 18:34:17.838105 22662 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:34:17.834398 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55110 (local address 127.19.194.2:38833)
0806 18:34:17.834773 (+ 375us) server_negotiation.cc:206] Beginning negotiation
0806 18:34:17.834777 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 18:34:17.834791 (+ 14us) server_negotiation.cc:407] Connection header received
0806 18:34:17.834835 (+ 44us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:34:17.834838 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:34:17.834887 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:34:17.834971 (+ 84us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 18:34:17.835242 (+ 271us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:34:17.836010 (+ 768us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:34:17.836884 (+ 874us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:34:17.837254 (+ 370us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:34:17.837372 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:34:17.837646 (+ 274us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 18:34:17.837759 (+ 113us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 18:34:17.837839 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0806 18:34:17.837912 (+ 73us) server_negotiation.cc:299] Negotiation successful
0806 18:34:17.837964 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":292}
I20260806 18:34:17.844256 22488 raft_consensus.cc:1261] T 19d56f4fd4f64470b6b0201f5586cb54 P c5fb6bfd8ec4401fb0f9b3cf91ae5ef3 [term 1 FOLLOWER]: Refusing update from remote peer b49bc56953ac498582361f7e613cec38: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 18:34:17.844390 22201 raft_consensus.cc:1261] T 19d56f4fd4f64470b6b0201f5586cb54 P 39d31d4b04c14099bf89458586ff793c [term 1 FOLLOWER]: Refusing update from remote peer b49bc56953ac498582361f7e613cec38: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 18:34:17.844647 22622 consensus_queue.cc:1035] T 19d56f4fd4f64470b6b0201f5586cb54 P b49bc56953ac498582361f7e613cec38 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c5fb6bfd8ec4401fb0f9b3cf91ae5ef3" member_type: VOTER last_known_addr { host: "127.19.194.3" port: 43573 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 18:34:17.844794 22566 consensus_queue.cc:1035] T 19d56f4fd4f64470b6b0201f5586cb54 P b49bc56953ac498582361f7e613cec38 [LEADER]: Connected to new peer: Peer: permanent_uuid: "39d31d4b04c14099bf89458586ff793c" member_type: VOTER last_known_addr { host: "127.19.194.1" port: 37593 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260806 18:34:17.859385 22620 tablet_replica.cc:1298] Aborted: operation has been aborted: cancelling pending write operations
I20260806 18:34:19.424364 22038 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:40694:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 18:34:19.424535 22038 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:40694:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 18:34:19.426854 22038 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 805a3be867fa43bdafe4c8974a803d51: Sending DeleteTablet for 3 replicas of tablet 19d56f4fd4f64470b6b0201f5586cb54
I20260806 18:34:19.427397 22180 tablet_service.cc:1514] Processing DeleteTablet for tablet 19d56f4fd4f64470b6b0201f5586cb54 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 18:34:19 UTC) from {username='kudu', principal='kudu/127.19.194.62@KRBTEST.COM'} at 127.0.0.1:49642
I20260806 18:34:19.427489 22324 tablet_service.cc:1514] Processing DeleteTablet for tablet 19d56f4fd4f64470b6b0201f5586cb54 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 18:34:19 UTC) from {username='kudu', principal='kudu/127.19.194.62@KRBTEST.COM'} at 127.0.0.1:55086
I20260806 18:34:19.427635 22468 tablet_service.cc:1514] Processing DeleteTablet for tablet 19d56f4fd4f64470b6b0201f5586cb54 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 18:34:19 UTC) from {username='kudu', principal='kudu/127.19.194.62@KRBTEST.COM'} at 127.0.0.1:36884
I20260806 18:34:19.427906 22704 tablet_replica.cc:330] T 19d56f4fd4f64470b6b0201f5586cb54 P b49bc56953ac498582361f7e613cec38: stopping tablet replica
I20260806 18:34:19.427959 22705 tablet_replica.cc:330] T 19d56f4fd4f64470b6b0201f5586cb54 P c5fb6bfd8ec4401fb0f9b3cf91ae5ef3: stopping tablet replica
I20260806 18:34:19.428063 22704 raft_consensus.cc:2229] T 19d56f4fd4f64470b6b0201f5586cb54 P b49bc56953ac498582361f7e613cec38 [term 1 LEADER]: Raft consensus shutting down.
I20260806 18:34:19.428153 22705 raft_consensus.cc:2229] T 19d56f4fd4f64470b6b0201f5586cb54 P c5fb6bfd8ec4401fb0f9b3cf91ae5ef3 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 18:34:19.428241 20232 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJW69tn/build/debug/bin/kudu with pid 22113
I20260806 18:34:19.428309 22705 raft_consensus.cc:2258] T 19d56f4fd4f64470b6b0201f5586cb54 P c5fb6bfd8ec4401fb0f9b3cf91ae5ef3 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 18:34:19.428284 22704 raft_consensus.cc:2258] T 19d56f4fd4f64470b6b0201f5586cb54 P b49bc56953ac498582361f7e613cec38 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 18:34:19.429893 22704 ts_tablet_manager.cc:1905] T 19d56f4fd4f64470b6b0201f5586cb54 P b49bc56953ac498582361f7e613cec38: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 18:34:19.430529 22705 ts_tablet_manager.cc:1905] T 19d56f4fd4f64470b6b0201f5586cb54 P c5fb6bfd8ec4401fb0f9b3cf91ae5ef3: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 18:34:19.432695 22704 ts_tablet_manager.cc:1918] T 19d56f4fd4f64470b6b0201f5586cb54 P b49bc56953ac498582361f7e613cec38: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260806 18:34:19.432790 22704 log.cc:1196] T 19d56f4fd4f64470b6b0201f5586cb54 P b49bc56953ac498582361f7e613cec38: Deleting WAL directory at /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786041246094603-20232-0/minicluster-data/ts-1/wal/wals/19d56f4fd4f64470b6b0201f5586cb54
I20260806 18:34:19.433063 22704 ts_tablet_manager.cc:1939] T 19d56f4fd4f64470b6b0201f5586cb54 P b49bc56953ac498582361f7e613cec38: Deleting consensus metadata
I20260806 18:34:19.433107 22705 ts_tablet_manager.cc:1918] T 19d56f4fd4f64470b6b0201f5586cb54 P c5fb6bfd8ec4401fb0f9b3cf91ae5ef3: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260806 18:34:19.433192 22705 log.cc:1196] T 19d56f4fd4f64470b6b0201f5586cb54 P c5fb6bfd8ec4401fb0f9b3cf91ae5ef3: Deleting WAL directory at /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786041246094603-20232-0/minicluster-data/ts-2/wal/wals/19d56f4fd4f64470b6b0201f5586cb54
I20260806 18:34:19.433466 22705 ts_tablet_manager.cc:1939] T 19d56f4fd4f64470b6b0201f5586cb54 P c5fb6bfd8ec4401fb0f9b3cf91ae5ef3: Deleting consensus metadata
I20260806 18:34:19.433773 22025 catalog_manager.cc:4841] TS b49bc56953ac498582361f7e613cec38 (127.19.194.2:38833): tablet 19d56f4fd4f64470b6b0201f5586cb54 (table test-table [id=d0aa5da933f140cea14c090e10ff6fca]) successfully deleted
I20260806 18:34:19.434252 22024 catalog_manager.cc:4841] TS c5fb6bfd8ec4401fb0f9b3cf91ae5ef3 (127.19.194.3:43573): tablet 19d56f4fd4f64470b6b0201f5586cb54 (table test-table [id=d0aa5da933f140cea14c090e10ff6fca]) successfully deleted
W20260806 18:34:19.440478 22024 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.19.194.1:37593 (error 108)
I20260806 18:34:19.440555 20232 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJW69tn/build/debug/bin/kudu with pid 22257
I20260806 18:34:19.442566 22707 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:34:19.442004 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.194.1:37593 (local address 127.0.0.1:34052)
0806 18:34:19.442317 (+ 313us) negotiation.cc:107] Waiting for socket to connect
0806 18:34:19.442418 (+ 101us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.194.1:37593: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":222,"thread_start_us":81,"threads_started":1}
W20260806 18:34:19.442713 22024 catalog_manager.cc:4584] TS 39d31d4b04c14099bf89458586ff793c (127.19.194.1:37593): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 19d56f4fd4f64470b6b0201f5586cb54: Network error: Client connection negotiation failed: client connection to 127.19.194.1:37593: connect: Connection refused (error 111)
I20260806 18:34:19.448951 20232 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJW69tn/build/debug/bin/kudu with pid 22401
I20260806 18:34:19.457170 20232 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJW69tn/build/debug/bin/kudu with pid 22007
2026-08-06T18:34:19Z chronyd exiting
[ OK ] SecurityITest.TxnSmokeWithDifferentUserTypes (6189 ms)
[ RUN ] SecurityITest.TestNoKerberosCredentials
Loading random data
Initializing database '/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786041246094603-20232-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 18:34:19 dist-test-slave-1xrh krb5kdc[22710](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 18:34:19 dist-test-slave-1xrh krb5kdc[22710](info): set up 2 sockets
Aug 06 18:34:19 dist-test-slave-1xrh krb5kdc[22710](info): commencing operation
krb5kdc: starting...
W20260806 18:34:20.523965 20232 mini_kdc.cc:121] Time spent starting KDC: real 1.043s user 0.002s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786041246094603-20232-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786041246094603-20232-0/krb5kdc/test-user.keytab.
Aug 06 18:34:20 dist-test-slave-1xrh krb5kdc[22710](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041260, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T18:34:20Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T18:34:20Z Disabled control of system clock
WARNING: no policy specified for kudu/127.19.194.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.19.194.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.19.194.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786041246094603-20232-0/krb5kdc/kudu_127.19.194.62.keytab.
Entry for principal kudu/127.19.194.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786041246094603-20232-0/krb5kdc/kudu_127.19.194.62.keytab.
I20260806 18:34:20.642942 20232 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJW69tn/build/debug/bin/kudu
/tmp/dist-test-taskJW69tn/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786041246094603-20232-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786041246094603-20232-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786041246094603-20232-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786041246094603-20232-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.19.194.62:45841
--webserver_interface=127.19.194.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786041246094603-20232-0/krb5kdc/kudu_127.19.194.62.keytab
--principal=kudu/127.19.194.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.19.194.20:33605
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.19.194.62:45841
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786041246094603-20232-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786041246094603-20232-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786041246094603-20232-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 18:34:20.750242 22723 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 18:34:20.750501 22723 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 18:34:20.750540 22723 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 18:34:20.754988 22723 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 18:34:20.755077 22723 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 18:34:20.755146 22723 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 18:34:20.755179 22723 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 18:34:20.755210 22723 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 18:34:20.761647 22723 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.194.20:33605
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786041246094603-20232-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786041246094603-20232-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.19.194.62:45841
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786041246094603-20232-0/krb5kdc/kudu_127.19.194.62.keytab
--principal=kudu/127.19.194.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.19.194.62:45841
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786041246094603-20232-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.19.194.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22723
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786041246094603-20232-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 17:59:13 UTC on ccycloud.quasar-wvsnzx.root.comops.site
build id 4571028
I20260806 18:34:20.763059 22723 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 18:34:20.764520 22723 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 18:34:20.774788 22732 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:34:20.774798 22728 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 18:34:20.775024 22723 server_base.cc:1034] running on GCE node
W20260806 18:34:20.775053 22730 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 18:34:20.775664 22723 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 18:34:20.777175 22723 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 18:34:20.778375 22723 hybrid_clock.cc:648] HybridClock initialized: now 1786041260778350 us; error 35 us; skew 500 ppm
Aug 06 18:34:20 dist-test-slave-1xrh krb5kdc[22710](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041260, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.194.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 18:34:20.781744 22723 init.cc:377] Logged in from keytab as kudu/127.19.194.62@KRBTEST.COM (short username kudu)
I20260806 18:34:20.782833 22723 webserver.cc:460] Webserver started at http://127.19.194.62:37973/ using document root <none> and password file <none>
I20260806 18:34:20.783452 22723 fs_manager.cc:359] Metadata directory not provided
I20260806 18:34:20.783541 22723 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 18:34:20.783841 22723 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 18:34:20.786670 22723 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786041246094603-20232-0/minicluster-data/master-0/data/instance:
uuid: "f7fd3a0807ba4c23a561b9f84a855e2f"
format_stamp: "Formatted at 2026-08-06 18:34:20 on dist-test-slave-1xrh"
server_key: "09ce128199c48b18c4c470b214e1c9ce"
server_key_iv: "8cd65c174ec10cac01b96e6873d001c2"
server_key_version: "encryptionkey@0"
I20260806 18:34:20.787463 22723 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786041246094603-20232-0/minicluster-data/master-0/wal/instance:
uuid: "f7fd3a0807ba4c23a561b9f84a855e2f"
format_stamp: "Formatted at 2026-08-06 18:34:20 on dist-test-slave-1xrh"
server_key: "09ce128199c48b18c4c470b214e1c9ce"
server_key_iv: "8cd65c174ec10cac01b96e6873d001c2"
server_key_version: "encryptionkey@0"
I20260806 18:34:20.792502 22723 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.001s
I20260806 18:34:20.795922 22739 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 18:34:20.797544 22723 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260806 18:34:20.797683 22723 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786041246094603-20232-0/minicluster-data/master-0/data,/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786041246094603-20232-0/minicluster-data/master-0/wal
uuid: "f7fd3a0807ba4c23a561b9f84a855e2f"
format_stamp: "Formatted at 2026-08-06 18:34:20 on dist-test-slave-1xrh"
server_key: "09ce128199c48b18c4c470b214e1c9ce"
server_key_iv: "8cd65c174ec10cac01b96e6873d001c2"
server_key_version: "encryptionkey@0"
I20260806 18:34:20.797840 22723 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786041246094603-20232-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786041246094603-20232-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786041246094603-20232-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 18:34:20.813642 22723 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 18:34:20.816730 22723 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 18:34:20.816884 22723 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 18:34:20.823722 22723 rpc_server.cc:307] RPC server started. Bound to: 127.19.194.62:45841
I20260806 18:34:20.823735 22794 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.194.62:45841 every 8 connection(s)
I20260806 18:34:20.824667 22723 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786041246094603-20232-0/minicluster-data/master-0/data/info.pb
I20260806 18:34:20.827284 22795 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 18:34:20.828819 20232 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJW69tn/build/debug/bin/kudu as pid 22723
I20260806 18:34:20.828900 20232 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786041246094603-20232-0/minicluster-data/master-0/wal/instance
I20260806 18:34:20.829180 20232 external_mini_cluster.cc:1428] Setting key 23e438abb3eea132eeee5a983ecbe3e4
I20260806 18:34:20.833741 22795 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f7fd3a0807ba4c23a561b9f84a855e2f: Bootstrap starting.
I20260806 18:34:20.836374 22795 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P f7fd3a0807ba4c23a561b9f84a855e2f: Neither blocks nor log segments found. Creating new log.
I20260806 18:34:20.837153 22795 log.cc:824] T 00000000000000000000000000000000 P f7fd3a0807ba4c23a561b9f84a855e2f: Log is configured to *not* fsync() on all Append() calls
Aug 06 18:34:20 dist-test-slave-1xrh krb5kdc[22710](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041260, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.19.194.62@KRBTEST.COM
I20260806 18:34:20.839299 22795 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f7fd3a0807ba4c23a561b9f84a855e2f: No bootstrap required, opened a new log
I20260806 18:34:20.842224 22795 raft_consensus.cc:348] T 00000000000000000000000000000000 P f7fd3a0807ba4c23a561b9f84a855e2f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f7fd3a0807ba4c23a561b9f84a855e2f" member_type: VOTER last_known_addr { host: "127.19.194.62" port: 45841 } }
I20260806 18:34:20.842564 22795 raft_consensus.cc:374] T 00000000000000000000000000000000 P f7fd3a0807ba4c23a561b9f84a855e2f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 18:34:20.842658 22795 raft_consensus.cc:729] T 00000000000000000000000000000000 P f7fd3a0807ba4c23a561b9f84a855e2f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f7fd3a0807ba4c23a561b9f84a855e2f, State: Initialized, Role: FOLLOWER
I20260806 18:34:20.843178 22795 consensus_queue.cc:260] T 00000000000000000000000000000000 P f7fd3a0807ba4c23a561b9f84a855e2f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f7fd3a0807ba4c23a561b9f84a855e2f" member_type: VOTER last_known_addr { host: "127.19.194.62" port: 45841 } }
I20260806 18:34:20.843346 22795 raft_consensus.cc:388] T 00000000000000000000000000000000 P f7fd3a0807ba4c23a561b9f84a855e2f [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 18:34:20.843417 22795 raft_consensus.cc:482] T 00000000000000000000000000000000 P f7fd3a0807ba4c23a561b9f84a855e2f [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 18:34:20.843518 22795 raft_consensus.cc:3037] T 00000000000000000000000000000000 P f7fd3a0807ba4c23a561b9f84a855e2f [term 0 FOLLOWER]: Advancing to term 1
I20260806 18:34:20.844377 22795 raft_consensus.cc:504] T 00000000000000000000000000000000 P f7fd3a0807ba4c23a561b9f84a855e2f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f7fd3a0807ba4c23a561b9f84a855e2f" member_type: VOTER last_known_addr { host: "127.19.194.62" port: 45841 } }
I20260806 18:34:20.844467 22798 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:34:20.830503 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56868 (local address 127.19.194.62:45841)
0806 18:34:20.830917 (+ 414us) server_negotiation.cc:206] Beginning negotiation
0806 18:34:20.830936 (+ 19us) server_negotiation.cc:399] Waiting for connection header
0806 18:34:20.830977 (+ 41us) server_negotiation.cc:407] Connection header received
0806 18:34:20.831887 (+ 910us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:34:20.831910 (+ 23us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:34:20.832318 (+ 408us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:34:20.832679 (+ 361us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:34:20.833308 (+ 629us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:34:20.834705 (+ 1397us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:34:20.835592 (+ 887us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:34:20.836300 (+ 708us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:34:20.836504 (+ 204us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:34:20.839237 (+ 2733us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:34:20.839241 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:34:20.839253 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 18:34:20.839278 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:34:20.841521 (+ 2243us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:34:20.842011 (+ 490us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:34:20.842015 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:34:20.842019 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:34:20.842121 (+ 102us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:34:20.842428 (+ 307us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:34:20.842430 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:34:20.842431 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:34:20.842924 (+ 493us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:34:20.843190 (+ 266us) server_negotiation.cc:1035] Waiting for connection context
0806 18:34:20.843472 (+ 282us) server_negotiation.cc:299] Negotiation successful
0806 18:34:20.843768 (+ 296us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":260,"thread_start_us":105,"threads_started":1}
I20260806 18:34:20.844760 22795 leader_election.cc:302] T 00000000000000000000000000000000 P f7fd3a0807ba4c23a561b9f84a855e2f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: f7fd3a0807ba4c23a561b9f84a855e2f; no voters:
I20260806 18:34:20.845074 22795 leader_election.cc:288] T 00000000000000000000000000000000 P f7fd3a0807ba4c23a561b9f84a855e2f [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 18:34:20.845197 22800 raft_consensus.cc:2781] T 00000000000000000000000000000000 P f7fd3a0807ba4c23a561b9f84a855e2f [term 1 FOLLOWER]: Leader election won for term 1
I20260806 18:34:20.845422 22800 raft_consensus.cc:686] T 00000000000000000000000000000000 P f7fd3a0807ba4c23a561b9f84a855e2f [term 1 LEADER]: Becoming Leader. State: Replica: f7fd3a0807ba4c23a561b9f84a855e2f, State: Running, Role: LEADER
I20260806 18:34:20.845706 22800 consensus_queue.cc:237] T 00000000000000000000000000000000 P f7fd3a0807ba4c23a561b9f84a855e2f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f7fd3a0807ba4c23a561b9f84a855e2f" member_type: VOTER last_known_addr { host: "127.19.194.62" port: 45841 } }
I20260806 18:34:20.846233 22795 sys_catalog.cc:564] T 00000000000000000000000000000000 P f7fd3a0807ba4c23a561b9f84a855e2f [sys.catalog]: configured and running, proceeding with master startup.
I20260806 18:34:20.847611 22802 sys_catalog.cc:455] T 00000000000000000000000000000000 P f7fd3a0807ba4c23a561b9f84a855e2f [sys.catalog]: SysCatalogTable state changed. Reason: New leader f7fd3a0807ba4c23a561b9f84a855e2f. Latest consensus state: current_term: 1 leader_uuid: "f7fd3a0807ba4c23a561b9f84a855e2f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f7fd3a0807ba4c23a561b9f84a855e2f" member_type: VOTER last_known_addr { host: "127.19.194.62" port: 45841 } } }
I20260806 18:34:20.847611 22801 sys_catalog.cc:455] T 00000000000000000000000000000000 P f7fd3a0807ba4c23a561b9f84a855e2f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "f7fd3a0807ba4c23a561b9f84a855e2f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f7fd3a0807ba4c23a561b9f84a855e2f" member_type: VOTER last_known_addr { host: "127.19.194.62" port: 45841 } } }
I20260806 18:34:20.847785 22802 sys_catalog.cc:458] T 00000000000000000000000000000000 P f7fd3a0807ba4c23a561b9f84a855e2f [sys.catalog]: This master's current role is: LEADER
I20260806 18:34:20.847821 22801 sys_catalog.cc:458] T 00000000000000000000000000000000 P f7fd3a0807ba4c23a561b9f84a855e2f [sys.catalog]: This master's current role is: LEADER
W20260806 18:34:20.850639 22815 catalog_manager.cc:1558] T 00000000000000000000000000000000 P f7fd3a0807ba4c23a561b9f84a855e2f: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260806 18:34:20.850742 22815 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260806 18:34:20.850812 22812 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 18:34:20.851625 22812 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 18:34:20.856827 22812 catalog_manager.cc:1347] Generated new cluster ID: 53fd45d9989c4eacb4a3f21117d1d58d
I20260806 18:34:20.856887 22812 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 18:34:20.864110 22812 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 18:34:20.865206 22812 catalog_manager.cc:1504] Loading token signing keys...
I20260806 18:34:20.869810 22812 catalog_manager.cc:5868] T 00000000000000000000000000000000 P f7fd3a0807ba4c23a561b9f84a855e2f: Generated new TSK 0
I20260806 18:34:20.870496 22812 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.19.194.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.19.194.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.19.194.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786041246094603-20232-0/krb5kdc/kudu_127.19.194.1.keytab.
Entry for principal kudu/127.19.194.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786041246094603-20232-0/krb5kdc/kudu_127.19.194.1.keytab.
I20260806 18:34:20.904121 20232 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJW69tn/build/debug/bin/kudu
/tmp/dist-test-taskJW69tn/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786041246094603-20232-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786041246094603-20232-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786041246094603-20232-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786041246094603-20232-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.19.194.1:0
--local_ip_for_outbound_sockets=127.19.194.1
--webserver_interface=127.19.194.1
--webserver_port=0
--tserver_master_addrs=127.19.194.62:45841
--keytab_file=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786041246094603-20232-0/krb5kdc/kudu_127.19.194.1.keytab
--principal=kudu/127.19.194.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.19.194.20:33605
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786041246094603-20232-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786041246094603-20232-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786041246094603-20232-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 18:34:21.016896 22822 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 18:34:21.017153 22822 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 18:34:21.017194 22822 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 18:34:21.021198 22822 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 18:34:21.021250 22822 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 18:34:21.021351 22822 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.19.194.1
I20260806 18:34:21.026391 22822 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.194.20:33605
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786041246094603-20232-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786041246094603-20232-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786041246094603-20232-0/krb5kdc/kudu_127.19.194.1.keytab
--principal=kudu/127.19.194.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.19.194.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786041246094603-20232-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.19.194.1
--webserver_port=0
--tserver_master_addrs=127.19.194.62:45841
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22822
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.19.194.1
--log_dir=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786041246094603-20232-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 17:59:13 UTC on ccycloud.quasar-wvsnzx.root.comops.site
build id 4571028
I20260806 18:34:21.027422 22822 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 18:34:21.028419 22822 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 18:34:21.037999 22830 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:34:21.038008 22828 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:34:21.038110 22827 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 18:34:21.038189 22822 server_base.cc:1034] running on GCE node
I20260806 18:34:21.038669 22822 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 18:34:21.039379 22822 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 18:34:21.040558 22822 hybrid_clock.cc:648] HybridClock initialized: now 1786041261040541 us; error 28 us; skew 500 ppm
Aug 06 18:34:21 dist-test-slave-1xrh krb5kdc[22710](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041261, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.194.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 18:34:21.043365 22822 init.cc:377] Logged in from keytab as kudu/127.19.194.1@KRBTEST.COM (short username kudu)
I20260806 18:34:21.044178 22822 webserver.cc:460] Webserver started at http://127.19.194.1:44719/ using document root <none> and password file <none>
I20260806 18:34:21.044721 22822 fs_manager.cc:359] Metadata directory not provided
I20260806 18:34:21.044813 22822 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 18:34:21.045042 22822 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 18:34:21.047060 22822 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786041246094603-20232-0/minicluster-data/ts-0/data/instance:
uuid: "e56b2a58437646c7ae6be15a07b4ca9c"
format_stamp: "Formatted at 2026-08-06 18:34:21 on dist-test-slave-1xrh"
server_key: "1d6ab4d2ef5574b5695197dfd5ee9076"
server_key_iv: "4b684e310c23978b189193687e16ab72"
server_key_version: "encryptionkey@0"
I20260806 18:34:21.047693 22822 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786041246094603-20232-0/minicluster-data/ts-0/wal/instance:
uuid: "e56b2a58437646c7ae6be15a07b4ca9c"
format_stamp: "Formatted at 2026-08-06 18:34:21 on dist-test-slave-1xrh"
server_key: "1d6ab4d2ef5574b5695197dfd5ee9076"
server_key_iv: "4b684e310c23978b189193687e16ab72"
server_key_version: "encryptionkey@0"
I20260806 18:34:21.051265 22822 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20260806 18:34:21.053591 22839 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 18:34:21.054737 22822 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260806 18:34:21.054867 22822 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786041246094603-20232-0/minicluster-data/ts-0/data,/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786041246094603-20232-0/minicluster-data/ts-0/wal
uuid: "e56b2a58437646c7ae6be15a07b4ca9c"
format_stamp: "Formatted at 2026-08-06 18:34:21 on dist-test-slave-1xrh"
server_key: "1d6ab4d2ef5574b5695197dfd5ee9076"
server_key_iv: "4b684e310c23978b189193687e16ab72"
server_key_version: "encryptionkey@0"
I20260806 18:34:21.054977 22822 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786041246094603-20232-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786041246094603-20232-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786041246094603-20232-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 18:34:21.071553 22822 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 18:34:21.074723 22822 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 18:34:21.074913 22822 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 18:34:21.075546 22822 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 18:34:21.076375 22822 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 18:34:21.076447 22822 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 18:34:21.076521 22822 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 18:34:21.076573 22822 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 18:34:21.085829 22822 rpc_server.cc:307] RPC server started. Bound to: 127.19.194.1:42073
I20260806 18:34:21.085889 22956 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.194.1:42073 every 8 connection(s)
I20260806 18:34:21.086635 22822 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786041246094603-20232-0/minicluster-data/ts-0/data/info.pb
I20260806 18:34:21.089917 20232 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJW69tn/build/debug/bin/kudu as pid 22822
I20260806 18:34:21.090049 20232 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786041246094603-20232-0/minicluster-data/ts-0/wal/instance
I20260806 18:34:21.090317 20232 external_mini_cluster.cc:1428] Setting key 37409ef8c57f5e9f437bbdf5ffc4ba5c
Aug 06 18:34:21 dist-test-slave-1xrh krb5kdc[22710](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041261, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.194.1@KRBTEST.COM for kudu/127.19.194.62@KRBTEST.COM
WARNING: no policy specified for kudu/127.19.194.2@KRBTEST.COM; defaulting to no policy
I20260806 18:34:21.099852 22798 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:34:21.088267 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.194.1:45709 (local address 127.19.194.62:45841)
0806 18:34:21.088395 (+ 128us) server_negotiation.cc:206] Beginning negotiation
0806 18:34:21.088399 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 18:34:21.089301 (+ 902us) server_negotiation.cc:407] Connection header received
0806 18:34:21.090350 (+ 1049us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:34:21.090353 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:34:21.090414 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:34:21.090517 (+ 103us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:34:21.091863 (+ 1346us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:34:21.092394 (+ 531us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:34:21.093516 (+ 1122us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:34:21.093749 (+ 233us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:34:21.093864 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:34:21.096322 (+ 2458us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:34:21.096325 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:34:21.096327 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 18:34:21.096349 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:34:21.097791 (+ 1442us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:34:21.098294 (+ 503us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:34:21.098303 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:34:21.098304 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:34:21.098383 (+ 79us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:34:21.098722 (+ 339us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:34:21.098725 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:34:21.098727 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:34:21.098912 (+ 185us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:34:21.099029 (+ 117us) server_negotiation.cc:1035] Waiting for connection context
0806 18:34:21.099549 (+ 520us) server_negotiation.cc:299] Negotiation successful
0806 18:34:21.099676 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260806 18:34:21.100600 22959 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:34:21.088592 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.194.62:45841 (local address 127.19.194.1:45709)
0806 18:34:21.089169 (+ 577us) negotiation.cc:107] Waiting for socket to connect
0806 18:34:21.089206 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0806 18:34:21.090135 (+ 929us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:34:21.090725 (+ 590us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:34:21.090736 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:34:21.091188 (+ 452us) client_negotiation.cc:189] Negotiated authn=SASL
0806 18:34:21.091704 (+ 516us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:34:21.091724 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:34:21.092535 (+ 811us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:34:21.092543 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:34:21.093388 (+ 845us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:34:21.093397 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:34:21.093901 (+ 504us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:34:21.093905 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:34:21.094084 (+ 179us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:34:21.094681 (+ 597us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 18:34:21.094697 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0806 18:34:21.096169 (+ 1472us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 18:34:21.097914 (+ 1745us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:34:21.097920 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:34:21.097932 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:34:21.098183 (+ 251us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:34:21.098476 (+ 293us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:34:21.098480 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:34:21.098482 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:34:21.098624 (+ 142us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:34:21.099021 (+ 397us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 18:34:21.099027 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 18:34:21.099323 (+ 296us) client_negotiation.cc:769] Sending connection context
0806 18:34:21.099504 (+ 181us) client_negotiation.cc:240] Negotiation successful
0806 18:34:21.099813 (+ 309us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":415,"thread_start_us":101,"threads_started":1}
I20260806 18:34:21.101722 22957 heartbeater.cc:347] Connected to a master server at 127.19.194.62:45841
I20260806 18:34:21.101922 22957 heartbeater.cc:464] Registering TS with master...
I20260806 18:34:21.102346 22957 heartbeater.cc:511] Master 127.19.194.62:45841 requested a full tablet report, sending...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.19.194.2@KRBTEST.COM" created.
I20260806 18:34:21.103938 22759 ts_manager.cc:194] Registered new tserver with Master: e56b2a58437646c7ae6be15a07b4ca9c (127.19.194.1:42073)
I20260806 18:34:21.105064 22759 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.19.194.1@KRBTEST.COM'} at 127.19.194.1:45709
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.19.194.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786041246094603-20232-0/krb5kdc/kudu_127.19.194.2.keytab.
Entry for principal kudu/127.19.194.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786041246094603-20232-0/krb5kdc/kudu_127.19.194.2.keytab.
I20260806 18:34:21.116384 20232 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJW69tn/build/debug/bin/kudu
/tmp/dist-test-taskJW69tn/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786041246094603-20232-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786041246094603-20232-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786041246094603-20232-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786041246094603-20232-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.19.194.2:0
--local_ip_for_outbound_sockets=127.19.194.2
--webserver_interface=127.19.194.2
--webserver_port=0
--tserver_master_addrs=127.19.194.62:45841
--keytab_file=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786041246094603-20232-0/krb5kdc/kudu_127.19.194.2.keytab
--principal=kudu/127.19.194.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.19.194.20:33605
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786041246094603-20232-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786041246094603-20232-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786041246094603-20232-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 18:34:21.225592 22962 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 18:34:21.225817 22962 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 18:34:21.225867 22962 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 18:34:21.229727 22962 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 18:34:21.229772 22962 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 18:34:21.229831 22962 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.19.194.2
I20260806 18:34:21.234659 22962 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.194.20:33605
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786041246094603-20232-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786041246094603-20232-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786041246094603-20232-0/krb5kdc/kudu_127.19.194.2.keytab
--principal=kudu/127.19.194.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.19.194.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786041246094603-20232-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.19.194.2
--webserver_port=0
--tserver_master_addrs=127.19.194.62:45841
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22962
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.19.194.2
--log_dir=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786041246094603-20232-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 17:59:13 UTC on ccycloud.quasar-wvsnzx.root.comops.site
build id 4571028
I20260806 18:34:21.235541 22962 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 18:34:21.236399 22962 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 18:34:21.245276 22968 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:34:21.245256 22967 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:34:21.245318 22970 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 18:34:21.245517 22962 server_base.cc:1034] running on GCE node
I20260806 18:34:21.245975 22962 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 18:34:21.246650 22962 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 18:34:21.247824 22962 hybrid_clock.cc:648] HybridClock initialized: now 1786041261247813 us; error 42 us; skew 500 ppm
Aug 06 18:34:21 dist-test-slave-1xrh krb5kdc[22710](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041261, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.194.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 18:34:21.250533 22962 init.cc:377] Logged in from keytab as kudu/127.19.194.2@KRBTEST.COM (short username kudu)
I20260806 18:34:21.251467 22962 webserver.cc:460] Webserver started at http://127.19.194.2:40177/ using document root <none> and password file <none>
I20260806 18:34:21.252017 22962 fs_manager.cc:359] Metadata directory not provided
I20260806 18:34:21.252107 22962 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 18:34:21.252341 22962 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 18:34:21.254311 22962 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786041246094603-20232-0/minicluster-data/ts-1/data/instance:
uuid: "96f34aa6e5b94089bc55942c6f58f139"
format_stamp: "Formatted at 2026-08-06 18:34:21 on dist-test-slave-1xrh"
server_key: "f9687317472e8193c88ae765b1e5de68"
server_key_iv: "84595d88cdeb7cf8e2ac6b7677ba7cd8"
server_key_version: "encryptionkey@0"
I20260806 18:34:21.254890 22962 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786041246094603-20232-0/minicluster-data/ts-1/wal/instance:
uuid: "96f34aa6e5b94089bc55942c6f58f139"
format_stamp: "Formatted at 2026-08-06 18:34:21 on dist-test-slave-1xrh"
server_key: "f9687317472e8193c88ae765b1e5de68"
server_key_iv: "84595d88cdeb7cf8e2ac6b7677ba7cd8"
server_key_version: "encryptionkey@0"
I20260806 18:34:21.258438 22962 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.002s sys 0.000s
I20260806 18:34:21.260898 22978 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 18:34:21.262055 22962 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.000s sys 0.001s
I20260806 18:34:21.262184 22962 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786041246094603-20232-0/minicluster-data/ts-1/data,/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786041246094603-20232-0/minicluster-data/ts-1/wal
uuid: "96f34aa6e5b94089bc55942c6f58f139"
format_stamp: "Formatted at 2026-08-06 18:34:21 on dist-test-slave-1xrh"
server_key: "f9687317472e8193c88ae765b1e5de68"
server_key_iv: "84595d88cdeb7cf8e2ac6b7677ba7cd8"
server_key_version: "encryptionkey@0"
I20260806 18:34:21.262287 22962 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786041246094603-20232-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786041246094603-20232-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786041246094603-20232-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 18:34:21.281172 22962 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 18:34:21.284121 22962 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 18:34:21.284309 22962 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 18:34:21.284986 22962 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 18:34:21.285758 22962 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 18:34:21.285802 22962 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 18:34:21.285876 22962 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 18:34:21.285909 22962 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 18:34:21.295029 22962 rpc_server.cc:307] RPC server started. Bound to: 127.19.194.2:37213
I20260806 18:34:21.295105 23094 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.194.2:37213 every 8 connection(s)
I20260806 18:34:21.295997 22962 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786041246094603-20232-0/minicluster-data/ts-1/data/info.pb
I20260806 18:34:21.302125 20232 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJW69tn/build/debug/bin/kudu as pid 22962
I20260806 18:34:21.302219 20232 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786041246094603-20232-0/minicluster-data/ts-1/wal/instance
I20260806 18:34:21.302537 20232 external_mini_cluster.cc:1428] Setting key d342593d6d04abb9e2a0cd4f9bcff442
Aug 06 18:34:21 dist-test-slave-1xrh krb5kdc[22710](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041261, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.194.2@KRBTEST.COM for kudu/127.19.194.62@KRBTEST.COM
I20260806 18:34:21.309975 22798 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:34:21.297658 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.194.2:58483 (local address 127.19.194.62:45841)
0806 18:34:21.297818 (+ 160us) server_negotiation.cc:206] Beginning negotiation
0806 18:34:21.297822 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 18:34:21.298889 (+ 1067us) server_negotiation.cc:407] Connection header received
0806 18:34:21.299767 (+ 878us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:34:21.299770 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:34:21.299817 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:34:21.299905 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:34:21.300950 (+ 1045us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:34:21.301446 (+ 496us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:34:21.302725 (+ 1279us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:34:21.303040 (+ 315us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:34:21.303168 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:34:21.306134 (+ 2966us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:34:21.306140 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:34:21.306145 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 18:34:21.306180 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:34:21.307793 (+ 1613us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:34:21.308272 (+ 479us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:34:21.308278 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:34:21.308282 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:34:21.308373 (+ 91us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:34:21.308701 (+ 328us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:34:21.308708 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:34:21.308712 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:34:21.308998 (+ 286us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:34:21.309116 (+ 118us) server_negotiation.cc:1035] Waiting for connection context
0806 18:34:21.309574 (+ 458us) server_negotiation.cc:299] Negotiation successful
0806 18:34:21.309724 (+ 150us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260806 18:34:21.310907 23097 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:34:21.298003 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.194.62:45841 (local address 127.19.194.2:58483)
0806 18:34:21.298744 (+ 741us) negotiation.cc:107] Waiting for socket to connect
0806 18:34:21.298782 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0806 18:34:21.299572 (+ 790us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:34:21.300046 (+ 474us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:34:21.300052 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:34:21.300387 (+ 335us) client_negotiation.cc:189] Negotiated authn=SASL
0806 18:34:21.300830 (+ 443us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:34:21.300842 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:34:21.301579 (+ 737us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:34:21.301581 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:34:21.302560 (+ 979us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:34:21.302587 (+ 27us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:34:21.303263 (+ 676us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:34:21.303268 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:34:21.303431 (+ 163us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:34:21.304143 (+ 712us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 18:34:21.304160 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0806 18:34:21.305932 (+ 1772us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 18:34:21.307907 (+ 1975us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:34:21.307912 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:34:21.307922 (+ 10us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:34:21.308168 (+ 246us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:34:21.308465 (+ 297us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:34:21.308467 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:34:21.308468 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:34:21.308598 (+ 130us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:34:21.309116 (+ 518us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 18:34:21.309120 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 18:34:21.309328 (+ 208us) client_negotiation.cc:769] Sending connection context
0806 18:34:21.309515 (+ 187us) client_negotiation.cc:240] Negotiation successful
0806 18:34:21.309867 (+ 352us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":568,"thread_start_us":105,"threads_started":1}
WARNING: no policy specified for kudu/127.19.194.3@KRBTEST.COM; defaulting to no policy
I20260806 18:34:21.315232 23095 heartbeater.cc:347] Connected to a master server at 127.19.194.62:45841
I20260806 18:34:21.315448 23095 heartbeater.cc:464] Registering TS with master...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.19.194.3@KRBTEST.COM" created.
I20260806 18:34:21.316052 23095 heartbeater.cc:511] Master 127.19.194.62:45841 requested a full tablet report, sending...
I20260806 18:34:21.317226 22759 ts_manager.cc:194] Registered new tserver with Master: 96f34aa6e5b94089bc55942c6f58f139 (127.19.194.2:37213)
I20260806 18:34:21.318089 22759 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.19.194.2@KRBTEST.COM'} at 127.19.194.2:58483
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.19.194.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786041246094603-20232-0/krb5kdc/kudu_127.19.194.3.keytab.
Entry for principal kudu/127.19.194.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786041246094603-20232-0/krb5kdc/kudu_127.19.194.3.keytab.
I20260806 18:34:21.331384 20232 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJW69tn/build/debug/bin/kudu
/tmp/dist-test-taskJW69tn/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786041246094603-20232-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786041246094603-20232-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786041246094603-20232-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786041246094603-20232-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.19.194.3:0
--local_ip_for_outbound_sockets=127.19.194.3
--webserver_interface=127.19.194.3
--webserver_port=0
--tserver_master_addrs=127.19.194.62:45841
--keytab_file=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786041246094603-20232-0/krb5kdc/kudu_127.19.194.3.keytab
--principal=kudu/127.19.194.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.19.194.20:33605
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786041246094603-20232-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786041246094603-20232-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786041246094603-20232-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 18:34:21.441323 23100 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 18:34:21.441552 23100 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 18:34:21.441592 23100 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 18:34:21.445623 23100 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 18:34:21.445676 23100 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 18:34:21.445786 23100 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.19.194.3
I20260806 18:34:21.450927 23100 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.194.20:33605
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786041246094603-20232-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786041246094603-20232-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786041246094603-20232-0/krb5kdc/kudu_127.19.194.3.keytab
--principal=kudu/127.19.194.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.19.194.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786041246094603-20232-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.19.194.3
--webserver_port=0
--tserver_master_addrs=127.19.194.62:45841
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23100
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.19.194.3
--log_dir=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786041246094603-20232-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 17:59:13 UTC on ccycloud.quasar-wvsnzx.root.comops.site
build id 4571028
I20260806 18:34:21.451895 23100 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 18:34:21.452816 23100 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 18:34:21.462082 23108 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:34:21.462150 23105 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:34:21.462095 23106 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 18:34:21.462219 23100 server_base.cc:1034] running on GCE node
I20260806 18:34:21.462764 23100 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 18:34:21.463419 23100 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 18:34:21.464582 23100 hybrid_clock.cc:648] HybridClock initialized: now 1786041261464568 us; error 28 us; skew 500 ppm
Aug 06 18:34:21 dist-test-slave-1xrh krb5kdc[22710](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041261, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.194.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 18:34:21.467338 23100 init.cc:377] Logged in from keytab as kudu/127.19.194.3@KRBTEST.COM (short username kudu)
I20260806 18:34:21.468186 23100 webserver.cc:460] Webserver started at http://127.19.194.3:46537/ using document root <none> and password file <none>
I20260806 18:34:21.468750 23100 fs_manager.cc:359] Metadata directory not provided
I20260806 18:34:21.468842 23100 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 18:34:21.469069 23100 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 18:34:21.471093 23100 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786041246094603-20232-0/minicluster-data/ts-2/data/instance:
uuid: "da5309a3e2ac4b6e93d91e1a47fe8101"
format_stamp: "Formatted at 2026-08-06 18:34:21 on dist-test-slave-1xrh"
server_key: "99db3da3e9202fd0d201c78903d8481e"
server_key_iv: "f2b96a73be82196013942ee11611d900"
server_key_version: "encryptionkey@0"
I20260806 18:34:21.471740 23100 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786041246094603-20232-0/minicluster-data/ts-2/wal/instance:
uuid: "da5309a3e2ac4b6e93d91e1a47fe8101"
format_stamp: "Formatted at 2026-08-06 18:34:21 on dist-test-slave-1xrh"
server_key: "99db3da3e9202fd0d201c78903d8481e"
server_key_iv: "f2b96a73be82196013942ee11611d900"
server_key_version: "encryptionkey@0"
I20260806 18:34:21.475427 23100 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20260806 18:34:21.477641 23115 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 18:34:21.478708 23100 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260806 18:34:21.478837 23100 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786041246094603-20232-0/minicluster-data/ts-2/data,/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786041246094603-20232-0/minicluster-data/ts-2/wal
uuid: "da5309a3e2ac4b6e93d91e1a47fe8101"
format_stamp: "Formatted at 2026-08-06 18:34:21 on dist-test-slave-1xrh"
server_key: "99db3da3e9202fd0d201c78903d8481e"
server_key_iv: "f2b96a73be82196013942ee11611d900"
server_key_version: "encryptionkey@0"
I20260806 18:34:21.478940 23100 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786041246094603-20232-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786041246094603-20232-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786041246094603-20232-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 18:34:21.518867 23100 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 18:34:21.521647 23100 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 18:34:21.521847 23100 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 18:34:21.522460 23100 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 18:34:21.523331 23100 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 18:34:21.523411 23100 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 18:34:21.523512 23100 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 18:34:21.523562 23100 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 18:34:21.532140 23100 rpc_server.cc:307] RPC server started. Bound to: 127.19.194.3:39573
I20260806 18:34:21.532174 23234 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.194.3:39573 every 8 connection(s)
I20260806 18:34:21.533018 23100 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786041246094603-20232-0/minicluster-data/ts-2/data/info.pb
I20260806 18:34:21.537266 20232 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJW69tn/build/debug/bin/kudu as pid 23100
I20260806 18:34:21.537434 20232 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786041246094603-20232-0/minicluster-data/ts-2/wal/instance
I20260806 18:34:21.537779 20232 external_mini_cluster.cc:1428] Setting key b3f11789c30a05faf82beda329f26234
Aug 06 18:34:21 dist-test-slave-1xrh krb5kdc[22710](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041261, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.194.3@KRBTEST.COM for kudu/127.19.194.62@KRBTEST.COM
I20260806 18:34:21.547513 22798 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:34:21.534836 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.194.3:55761 (local address 127.19.194.62:45841)
0806 18:34:21.535011 (+ 175us) server_negotiation.cc:206] Beginning negotiation
0806 18:34:21.535015 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 18:34:21.535768 (+ 753us) server_negotiation.cc:407] Connection header received
0806 18:34:21.537275 (+ 1507us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:34:21.537277 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:34:21.537327 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:34:21.537424 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:34:21.538974 (+ 1550us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:34:21.539653 (+ 679us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:34:21.540998 (+ 1345us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:34:21.541206 (+ 208us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:34:21.541284 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:34:21.543944 (+ 2660us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:34:21.543947 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:34:21.543949 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 18:34:21.543974 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:34:21.545283 (+ 1309us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:34:21.545784 (+ 501us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:34:21.545787 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:34:21.545788 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:34:21.545892 (+ 104us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:34:21.546303 (+ 411us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:34:21.546306 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:34:21.546308 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:34:21.546491 (+ 183us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:34:21.546582 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0806 18:34:21.547242 (+ 660us) server_negotiation.cc:299] Negotiation successful
0806 18:34:21.547379 (+ 137us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":74}
I20260806 18:34:21.548313 23237 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:34:21.535100 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.194.62:45841 (local address 127.19.194.3:55761)
0806 18:34:21.535610 (+ 510us) negotiation.cc:107] Waiting for socket to connect
0806 18:34:21.535655 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0806 18:34:21.537018 (+ 1363us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:34:21.537649 (+ 631us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:34:21.537660 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:34:21.538172 (+ 512us) client_negotiation.cc:189] Negotiated authn=SASL
0806 18:34:21.538789 (+ 617us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:34:21.538810 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:34:21.539833 (+ 1023us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:34:21.539837 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:34:21.540883 (+ 1046us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:34:21.540894 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:34:21.541338 (+ 444us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:34:21.541341 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:34:21.541483 (+ 142us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:34:21.542152 (+ 669us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 18:34:21.542169 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0806 18:34:21.543752 (+ 1583us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 18:34:21.545398 (+ 1646us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:34:21.545403 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:34:21.545415 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:34:21.545681 (+ 266us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:34:21.545989 (+ 308us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:34:21.545991 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:34:21.545993 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:34:21.546198 (+ 205us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:34:21.546602 (+ 404us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 18:34:21.546608 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 18:34:21.546936 (+ 328us) client_negotiation.cc:769] Sending connection context
0806 18:34:21.547114 (+ 178us) client_negotiation.cc:240] Negotiation successful
0806 18:34:21.547486 (+ 372us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":275,"thread_start_us":108,"threads_started":1}
I20260806 18:34:21.549517 23235 heartbeater.cc:347] Connected to a master server at 127.19.194.62:45841
I20260806 18:34:21.549751 23235 heartbeater.cc:464] Registering TS with master...
I20260806 18:34:21.550189 23235 heartbeater.cc:511] Master 127.19.194.62:45841 requested a full tablet report, sending...
I20260806 18:34:21.551337 22759 ts_manager.cc:194] Registered new tserver with Master: da5309a3e2ac4b6e93d91e1a47fe8101 (127.19.194.3:39573)
I20260806 18:34:21.552066 22759 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.19.194.3@KRBTEST.COM'} at 127.19.194.3:55761
I20260806 18:34:21.552080 20232 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
W20260806 18:34:21.567014 22798 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 18:34:21.565529 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56872 (local address 127.19.194.62:45841)
0806 18:34:21.565688 (+ 159us) server_negotiation.cc:206] Beginning negotiation
0806 18:34:21.565693 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 18:34:21.565854 (+ 161us) server_negotiation.cc:407] Connection header received
0806 18:34:21.565986 (+ 132us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:34:21.565989 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:34:21.566031 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:34:21.566149 (+ 118us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:34:21.566856 (+ 707us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:56872: BlockingRecv error: recv got EOF from 127.0.0.1:56872 (error 108)
Metrics: {"server-negotiator.queue_time_us":62}
I20260806 18:34:21.567951 20232 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJW69tn/build/debug/bin/kudu with pid 22822
I20260806 18:34:21.574265 20232 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJW69tn/build/debug/bin/kudu with pid 22962
I20260806 18:34:21.580561 20232 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJW69tn/build/debug/bin/kudu with pid 23100
I20260806 18:34:21.587533 20232 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJW69tn/build/debug/bin/kudu with pid 22723
2026-08-06T18:34:21Z chronyd exiting
[ OK ] SecurityITest.TestNoKerberosCredentials (2129 ms)
[ RUN ] SecurityITest.SaslPlainFallback
Loading random data
Initializing database '/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786041246094603-20232-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 18:34:21 dist-test-slave-1xrh krb5kdc[23248](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 18:34:21 dist-test-slave-1xrh krb5kdc[23248](info): set up 2 sockets
Aug 06 18:34:21 dist-test-slave-1xrh krb5kdc[23248](info): commencing operation
krb5kdc: starting...
W20260806 18:34:22.636570 20232 mini_kdc.cc:121] Time spent starting KDC: real 1.027s user 0.001s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786041246094603-20232-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786041246094603-20232-0/krb5kdc/test-user.keytab.
Aug 06 18:34:22 dist-test-slave-1xrh krb5kdc[23248](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041262, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T18:34:22Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T18:34:22Z Disabled control of system clock
WARNING: no policy specified for kudu/127.19.194.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.19.194.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.19.194.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786041246094603-20232-0/krb5kdc/kudu_127.19.194.62.keytab.
Entry for principal kudu/127.19.194.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786041246094603-20232-0/krb5kdc/kudu_127.19.194.62.keytab.
I20260806 18:34:22.758153 20232 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJW69tn/build/debug/bin/kudu
/tmp/dist-test-taskJW69tn/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786041246094603-20232-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786041246094603-20232-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786041246094603-20232-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786041246094603-20232-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.19.194.62:41021
--webserver_interface=127.19.194.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786041246094603-20232-0/krb5kdc/kudu_127.19.194.62.keytab
--principal=kudu/127.19.194.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.19.194.20:41091
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.19.194.62:41021
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc-authentication=optional
--user-acl=* with env {KRB5CCNAME=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786041246094603-20232-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786041246094603-20232-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786041246094603-20232-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 18:34:22.864240 23261 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 18:34:22.864602 23261 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 18:34:22.864681 23261 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 18:34:22.868618 23261 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 18:34:22.868695 23261 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 18:34:22.868733 23261 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 18:34:22.868768 23261 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 18:34:22.868805 23261 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 18:34:22.873796 23261 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.194.20:41091
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786041246094603-20232-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786041246094603-20232-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.19.194.62:41021
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786041246094603-20232-0/krb5kdc/kudu_127.19.194.62.keytab
--principal=kudu/127.19.194.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.19.194.62:41021
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786041246094603-20232-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.19.194.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23261
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786041246094603-20232-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 17:59:13 UTC on ccycloud.quasar-wvsnzx.root.comops.site
build id 4571028
I20260806 18:34:22.874761 23261 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 18:34:22.875736 23261 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 18:34:22.883997 23267 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:34:22.884033 23266 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 18:34:22.884127 23261 server_base.cc:1034] running on GCE node
W20260806 18:34:22.884205 23269 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 18:34:22.884781 23261 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 18:34:22.885850 23261 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 18:34:22.887042 23261 hybrid_clock.cc:648] HybridClock initialized: now 1786041262887025 us; error 32 us; skew 500 ppm
Aug 06 18:34:22 dist-test-slave-1xrh krb5kdc[23248](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041262, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.194.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 18:34:22.895309 23261 init.cc:377] Logged in from keytab as kudu/127.19.194.62@KRBTEST.COM (short username kudu)
I20260806 18:34:22.896207 23261 webserver.cc:460] Webserver started at http://127.19.194.62:44257/ using document root <none> and password file <none>
I20260806 18:34:22.896750 23261 fs_manager.cc:359] Metadata directory not provided
I20260806 18:34:22.896838 23261 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 18:34:22.897071 23261 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 18:34:22.899087 23261 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786041246094603-20232-0/minicluster-data/master-0/data/instance:
uuid: "3154ea299a974f9893ee82ed31e458c8"
format_stamp: "Formatted at 2026-08-06 18:34:22 on dist-test-slave-1xrh"
server_key: "1b3ce3c960898f3264175326b62d706d"
server_key_iv: "d4cf1fbdbc1ea781c1c7ad55ba1edb49"
server_key_version: "encryptionkey@0"
I20260806 18:34:22.899744 23261 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786041246094603-20232-0/minicluster-data/master-0/wal/instance:
uuid: "3154ea299a974f9893ee82ed31e458c8"
format_stamp: "Formatted at 2026-08-06 18:34:22 on dist-test-slave-1xrh"
server_key: "1b3ce3c960898f3264175326b62d706d"
server_key_iv: "d4cf1fbdbc1ea781c1c7ad55ba1edb49"
server_key_version: "encryptionkey@0"
I20260806 18:34:22.903405 23261 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20260806 18:34:22.905695 23276 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 18:34:22.906854 23261 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260806 18:34:22.906991 23261 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786041246094603-20232-0/minicluster-data/master-0/data,/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786041246094603-20232-0/minicluster-data/master-0/wal
uuid: "3154ea299a974f9893ee82ed31e458c8"
format_stamp: "Formatted at 2026-08-06 18:34:22 on dist-test-slave-1xrh"
server_key: "1b3ce3c960898f3264175326b62d706d"
server_key_iv: "d4cf1fbdbc1ea781c1c7ad55ba1edb49"
server_key_version: "encryptionkey@0"
I20260806 18:34:22.907097 23261 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786041246094603-20232-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786041246094603-20232-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786041246094603-20232-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 18:34:22.928362 23261 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 18:34:22.930994 23261 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 18:34:22.931208 23261 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 18:34:22.938151 23261 rpc_server.cc:307] RPC server started. Bound to: 127.19.194.62:41021
I20260806 18:34:22.938454 23332 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.194.62:41021 every 8 connection(s)
I20260806 18:34:22.939082 23261 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786041246094603-20232-0/minicluster-data/master-0/data/info.pb
I20260806 18:34:22.941694 23333 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 18:34:22.944585 20232 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJW69tn/build/debug/bin/kudu as pid 23261
I20260806 18:34:22.944701 20232 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786041246094603-20232-0/minicluster-data/master-0/wal/instance
I20260806 18:34:22.945000 20232 external_mini_cluster.cc:1428] Setting key 3116c9e34aa3a5184e3d790c9c075a47
I20260806 18:34:22.947594 23333 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3154ea299a974f9893ee82ed31e458c8: Bootstrap starting.
I20260806 18:34:22.950067 23333 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 3154ea299a974f9893ee82ed31e458c8: Neither blocks nor log segments found. Creating new log.
I20260806 18:34:22.950918 23333 log.cc:824] T 00000000000000000000000000000000 P 3154ea299a974f9893ee82ed31e458c8: Log is configured to *not* fsync() on all Append() calls
I20260806 18:34:22.953209 23333 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3154ea299a974f9893ee82ed31e458c8: No bootstrap required, opened a new log
Aug 06 18:34:22 dist-test-slave-1xrh krb5kdc[23248](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041262, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.19.194.62@KRBTEST.COM
I20260806 18:34:22.956178 23333 raft_consensus.cc:348] T 00000000000000000000000000000000 P 3154ea299a974f9893ee82ed31e458c8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3154ea299a974f9893ee82ed31e458c8" member_type: VOTER last_known_addr { host: "127.19.194.62" port: 41021 } }
I20260806 18:34:22.956415 23333 raft_consensus.cc:374] T 00000000000000000000000000000000 P 3154ea299a974f9893ee82ed31e458c8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 18:34:22.956490 23333 raft_consensus.cc:729] T 00000000000000000000000000000000 P 3154ea299a974f9893ee82ed31e458c8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3154ea299a974f9893ee82ed31e458c8, State: Initialized, Role: FOLLOWER
I20260806 18:34:22.956908 23333 consensus_queue.cc:260] T 00000000000000000000000000000000 P 3154ea299a974f9893ee82ed31e458c8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3154ea299a974f9893ee82ed31e458c8" member_type: VOTER last_known_addr { host: "127.19.194.62" port: 41021 } }
I20260806 18:34:22.957054 23333 raft_consensus.cc:388] T 00000000000000000000000000000000 P 3154ea299a974f9893ee82ed31e458c8 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 18:34:22.957140 23333 raft_consensus.cc:482] T 00000000000000000000000000000000 P 3154ea299a974f9893ee82ed31e458c8 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 18:34:22.957293 23333 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 3154ea299a974f9893ee82ed31e458c8 [term 0 FOLLOWER]: Advancing to term 1
I20260806 18:34:22.958139 23333 raft_consensus.cc:504] T 00000000000000000000000000000000 P 3154ea299a974f9893ee82ed31e458c8 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3154ea299a974f9893ee82ed31e458c8" member_type: VOTER last_known_addr { host: "127.19.194.62" port: 41021 } }
I20260806 18:34:22.958496 23333 leader_election.cc:302] T 00000000000000000000000000000000 P 3154ea299a974f9893ee82ed31e458c8 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 3154ea299a974f9893ee82ed31e458c8; no voters:
I20260806 18:34:22.958866 23333 leader_election.cc:288] T 00000000000000000000000000000000 P 3154ea299a974f9893ee82ed31e458c8 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 18:34:22.958990 23338 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 3154ea299a974f9893ee82ed31e458c8 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 18:34:22.959270 23338 raft_consensus.cc:686] T 00000000000000000000000000000000 P 3154ea299a974f9893ee82ed31e458c8 [term 1 LEADER]: Becoming Leader. State: Replica: 3154ea299a974f9893ee82ed31e458c8, State: Running, Role: LEADER
I20260806 18:34:22.959625 23338 consensus_queue.cc:237] T 00000000000000000000000000000000 P 3154ea299a974f9893ee82ed31e458c8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3154ea299a974f9893ee82ed31e458c8" member_type: VOTER last_known_addr { host: "127.19.194.62" port: 41021 } }
I20260806 18:34:22.960119 23333 sys_catalog.cc:564] T 00000000000000000000000000000000 P 3154ea299a974f9893ee82ed31e458c8 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 18:34:22.960178 23336 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:34:22.946254 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38130 (local address 127.19.194.62:41021)
0806 18:34:22.946662 (+ 408us) server_negotiation.cc:206] Beginning negotiation
0806 18:34:22.946678 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0806 18:34:22.946712 (+ 34us) server_negotiation.cc:407] Connection header received
0806 18:34:22.947321 (+ 609us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:34:22.947337 (+ 16us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:34:22.947675 (+ 338us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:34:22.947961 (+ 286us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:34:22.948886 (+ 925us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:34:22.949838 (+ 952us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:34:22.951020 (+ 1182us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:34:22.951622 (+ 602us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:34:22.951777 (+ 155us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:34:22.954452 (+ 2675us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:34:22.954457 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:34:22.954480 (+ 23us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 18:34:22.954508 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:34:22.956921 (+ 2413us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:34:22.957436 (+ 515us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:34:22.957442 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:34:22.957448 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:34:22.957569 (+ 121us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:34:22.957857 (+ 288us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:34:22.957861 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:34:22.957863 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:34:22.958238 (+ 375us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:34:22.958426 (+ 188us) server_negotiation.cc:1035] Waiting for connection context
0806 18:34:22.958743 (+ 317us) server_negotiation.cc:299] Negotiation successful
0806 18:34:22.959251 (+ 508us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":262,"thread_start_us":119,"threads_started":1}
I20260806 18:34:22.960732 23338 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3154ea299a974f9893ee82ed31e458c8 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 3154ea299a974f9893ee82ed31e458c8. Latest consensus state: current_term: 1 leader_uuid: "3154ea299a974f9893ee82ed31e458c8" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3154ea299a974f9893ee82ed31e458c8" member_type: VOTER last_known_addr { host: "127.19.194.62" port: 41021 } } }
I20260806 18:34:22.961499 23338 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3154ea299a974f9893ee82ed31e458c8 [sys.catalog]: This master's current role is: LEADER
I20260806 18:34:22.961390 23340 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3154ea299a974f9893ee82ed31e458c8 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "3154ea299a974f9893ee82ed31e458c8" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3154ea299a974f9893ee82ed31e458c8" member_type: VOTER last_known_addr { host: "127.19.194.62" port: 41021 } } }
I20260806 18:34:22.961792 23340 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3154ea299a974f9893ee82ed31e458c8 [sys.catalog]: This master's current role is: LEADER
I20260806 18:34:22.962471 23348 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 18:34:22.963339 20232 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260806 18:34:22.965520 23348 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 18:34:22.971513 23348 catalog_manager.cc:1347] Generated new cluster ID: 5d658278469742ceadd4cfb09750dc5c
I20260806 18:34:22.971575 23348 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 18:34:22.986847 23336 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:34:22.981099 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38136 (local address 127.19.194.62:41021)
0806 18:34:22.981204 (+ 105us) server_negotiation.cc:206] Beginning negotiation
0806 18:34:22.981208 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 18:34:22.981485 (+ 277us) server_negotiation.cc:407] Connection header received
0806 18:34:22.981603 (+ 118us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:34:22.981607 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:34:22.981661 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:34:22.981756 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:34:22.982319 (+ 563us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:34:22.983173 (+ 854us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:34:22.984332 (+ 1159us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:34:22.984708 (+ 376us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:34:22.984814 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:34:22.985729 (+ 915us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:34:22.985733 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:34:22.985735 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 18:34:22.986348 (+ 613us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:34:22.986385 (+ 37us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 18:34:22.986407 (+ 22us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:34:22.986503 (+ 96us) server_negotiation.cc:1035] Waiting for connection context
0806 18:34:22.986621 (+ 118us) server_negotiation.cc:299] Negotiation successful
0806 18:34:22.986680 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260806 18:34:22.997609 23348 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 18:34:22.998601 23348 catalog_manager.cc:1504] Loading token signing keys...
I20260806 18:34:23.004715 23348 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 3154ea299a974f9893ee82ed31e458c8: Generated new TSK 0
I20260806 18:34:23.005421 23348 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 18:34:23.015452 20232 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJW69tn/build/debug/bin/kudu with pid 23261
2026-08-06T18:34:23Z chronyd exiting
[ OK ] SecurityITest.SaslPlainFallback (1426 ms)
[ RUN ] SecurityITest.TestUnauthorizedClientKerberosCredentials
Loading random data
Initializing database '/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786041246094603-20232-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 18:34:23 dist-test-slave-1xrh krb5kdc[23367](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 18:34:23 dist-test-slave-1xrh krb5kdc[23367](info): set up 2 sockets
Aug 06 18:34:23 dist-test-slave-1xrh krb5kdc[23367](info): commencing operation
krb5kdc: starting...
W20260806 18:34:24.069619 20232 mini_kdc.cc:121] Time spent starting KDC: real 1.033s user 0.001s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786041246094603-20232-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786041246094603-20232-0/krb5kdc/test-user.keytab.
Aug 06 18:34:24 dist-test-slave-1xrh krb5kdc[23367](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041264, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T18:34:24Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T18:34:24Z Disabled control of system clock
WARNING: no policy specified for kudu/127.19.194.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.19.194.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.19.194.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786041246094603-20232-0/krb5kdc/kudu_127.19.194.62.keytab.
Entry for principal kudu/127.19.194.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786041246094603-20232-0/krb5kdc/kudu_127.19.194.62.keytab.
I20260806 18:34:24.192368 20232 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJW69tn/build/debug/bin/kudu
/tmp/dist-test-taskJW69tn/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786041246094603-20232-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786041246094603-20232-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786041246094603-20232-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786041246094603-20232-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.19.194.62:46845
--webserver_interface=127.19.194.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786041246094603-20232-0/krb5kdc/kudu_127.19.194.62.keytab
--principal=kudu/127.19.194.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.19.194.20:33261
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.19.194.62:46845
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786041246094603-20232-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786041246094603-20232-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786041246094603-20232-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 18:34:24.301504 23380 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 18:34:24.301748 23380 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 18:34:24.301885 23380 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 18:34:24.305881 23380 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 18:34:24.305931 23380 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 18:34:24.305950 23380 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 18:34:24.305965 23380 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 18:34:24.305980 23380 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 18:34:24.311300 23380 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.194.20:33261
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786041246094603-20232-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786041246094603-20232-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.19.194.62:46845
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786041246094603-20232-0/krb5kdc/kudu_127.19.194.62.keytab
--principal=kudu/127.19.194.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.19.194.62:46845
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786041246094603-20232-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.19.194.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23380
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786041246094603-20232-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 17:59:13 UTC on ccycloud.quasar-wvsnzx.root.comops.site
build id 4571028
I20260806 18:34:24.312316 23380 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 18:34:24.313350 23380 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 18:34:24.321651 23388 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:34:24.321674 23385 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 18:34:24.321743 23380 server_base.cc:1034] running on GCE node
W20260806 18:34:24.321691 23386 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 18:34:24.322328 23380 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 18:34:24.323503 23380 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 18:34:24.324664 23380 hybrid_clock.cc:648] HybridClock initialized: now 1786041264324630 us; error 47 us; skew 500 ppm
Aug 06 18:34:24 dist-test-slave-1xrh krb5kdc[23367](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041264, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.194.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 18:34:24.327379 23380 init.cc:377] Logged in from keytab as kudu/127.19.194.62@KRBTEST.COM (short username kudu)
I20260806 18:34:24.328158 23380 webserver.cc:460] Webserver started at http://127.19.194.62:45349/ using document root <none> and password file <none>
I20260806 18:34:24.328651 23380 fs_manager.cc:359] Metadata directory not provided
I20260806 18:34:24.328711 23380 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 18:34:24.328879 23380 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 18:34:24.330792 23380 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786041246094603-20232-0/minicluster-data/master-0/data/instance:
uuid: "d321f16d40b14d33aa6b39ec0df23568"
format_stamp: "Formatted at 2026-08-06 18:34:24 on dist-test-slave-1xrh"
server_key: "6621118f25b6cced724c6dccfba94579"
server_key_iv: "002dbcc59dfd13ecc32975a80560abd0"
server_key_version: "encryptionkey@0"
I20260806 18:34:24.331364 23380 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786041246094603-20232-0/minicluster-data/master-0/wal/instance:
uuid: "d321f16d40b14d33aa6b39ec0df23568"
format_stamp: "Formatted at 2026-08-06 18:34:24 on dist-test-slave-1xrh"
server_key: "6621118f25b6cced724c6dccfba94579"
server_key_iv: "002dbcc59dfd13ecc32975a80560abd0"
server_key_version: "encryptionkey@0"
I20260806 18:34:24.334884 23380 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.002s sys 0.000s
I20260806 18:34:24.337211 23396 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 18:34:24.338363 23380 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260806 18:34:24.338486 23380 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786041246094603-20232-0/minicluster-data/master-0/data,/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786041246094603-20232-0/minicluster-data/master-0/wal
uuid: "d321f16d40b14d33aa6b39ec0df23568"
format_stamp: "Formatted at 2026-08-06 18:34:24 on dist-test-slave-1xrh"
server_key: "6621118f25b6cced724c6dccfba94579"
server_key_iv: "002dbcc59dfd13ecc32975a80560abd0"
server_key_version: "encryptionkey@0"
I20260806 18:34:24.338587 23380 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786041246094603-20232-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786041246094603-20232-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786041246094603-20232-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 18:34:24.350852 23380 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 18:34:24.353521 23380 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 18:34:24.353718 23380 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 18:34:24.360752 23380 rpc_server.cc:307] RPC server started. Bound to: 127.19.194.62:46845
I20260806 18:34:24.361254 23448 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.194.62:46845 every 8 connection(s)
I20260806 18:34:24.361713 23380 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786041246094603-20232-0/minicluster-data/master-0/data/info.pb
I20260806 18:34:24.364544 23449 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 18:34:24.367959 20232 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJW69tn/build/debug/bin/kudu as pid 23380
I20260806 18:34:24.368103 20232 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786041246094603-20232-0/minicluster-data/master-0/wal/instance
I20260806 18:34:24.368384 20232 external_mini_cluster.cc:1428] Setting key 4c0b3ba50f9ce6c7586647e6d1836f53
I20260806 18:34:24.370850 23449 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d321f16d40b14d33aa6b39ec0df23568: Bootstrap starting.
I20260806 18:34:24.373553 23449 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P d321f16d40b14d33aa6b39ec0df23568: Neither blocks nor log segments found. Creating new log.
I20260806 18:34:24.374538 23449 log.cc:824] T 00000000000000000000000000000000 P d321f16d40b14d33aa6b39ec0df23568: Log is configured to *not* fsync() on all Append() calls
I20260806 18:34:24.376896 23449 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d321f16d40b14d33aa6b39ec0df23568: No bootstrap required, opened a new log
Aug 06 18:34:24 dist-test-slave-1xrh krb5kdc[23367](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041264, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.19.194.62@KRBTEST.COM
I20260806 18:34:24.380890 23449 raft_consensus.cc:348] T 00000000000000000000000000000000 P d321f16d40b14d33aa6b39ec0df23568 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d321f16d40b14d33aa6b39ec0df23568" member_type: VOTER last_known_addr { host: "127.19.194.62" port: 46845 } }
I20260806 18:34:24.381187 23449 raft_consensus.cc:374] T 00000000000000000000000000000000 P d321f16d40b14d33aa6b39ec0df23568 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 18:34:24.381253 23449 raft_consensus.cc:729] T 00000000000000000000000000000000 P d321f16d40b14d33aa6b39ec0df23568 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d321f16d40b14d33aa6b39ec0df23568, State: Initialized, Role: FOLLOWER
I20260806 18:34:24.381738 23449 consensus_queue.cc:260] T 00000000000000000000000000000000 P d321f16d40b14d33aa6b39ec0df23568 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d321f16d40b14d33aa6b39ec0df23568" member_type: VOTER last_known_addr { host: "127.19.194.62" port: 46845 } }
I20260806 18:34:24.381916 23449 raft_consensus.cc:388] T 00000000000000000000000000000000 P d321f16d40b14d33aa6b39ec0df23568 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 18:34:24.381997 23449 raft_consensus.cc:482] T 00000000000000000000000000000000 P d321f16d40b14d33aa6b39ec0df23568 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 18:34:24.382086 23449 raft_consensus.cc:3037] T 00000000000000000000000000000000 P d321f16d40b14d33aa6b39ec0df23568 [term 0 FOLLOWER]: Advancing to term 1
I20260806 18:34:24.383212 23449 raft_consensus.cc:504] T 00000000000000000000000000000000 P d321f16d40b14d33aa6b39ec0df23568 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d321f16d40b14d33aa6b39ec0df23568" member_type: VOTER last_known_addr { host: "127.19.194.62" port: 46845 } }
I20260806 18:34:24.383554 23449 leader_election.cc:302] T 00000000000000000000000000000000 P d321f16d40b14d33aa6b39ec0df23568 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: d321f16d40b14d33aa6b39ec0df23568; no voters:
I20260806 18:34:24.383896 23449 leader_election.cc:288] T 00000000000000000000000000000000 P d321f16d40b14d33aa6b39ec0df23568 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 18:34:24.384009 23454 raft_consensus.cc:2781] T 00000000000000000000000000000000 P d321f16d40b14d33aa6b39ec0df23568 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 18:34:24.384253 23454 raft_consensus.cc:686] T 00000000000000000000000000000000 P d321f16d40b14d33aa6b39ec0df23568 [term 1 LEADER]: Becoming Leader. State: Replica: d321f16d40b14d33aa6b39ec0df23568, State: Running, Role: LEADER
I20260806 18:34:24.384253 23452 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:34:24.369692 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40706 (local address 127.19.194.62:46845)
0806 18:34:24.370092 (+ 400us) server_negotiation.cc:206] Beginning negotiation
0806 18:34:24.370107 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0806 18:34:24.370141 (+ 34us) server_negotiation.cc:407] Connection header received
0806 18:34:24.370719 (+ 578us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:34:24.370734 (+ 15us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:34:24.371092 (+ 358us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:34:24.371413 (+ 321us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:34:24.372422 (+ 1009us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:34:24.373383 (+ 961us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:34:24.374764 (+ 1381us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:34:24.375459 (+ 695us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:34:24.375581 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:34:24.378293 (+ 2712us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:34:24.378300 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:34:24.378317 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 18:34:24.378350 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:34:24.380932 (+ 2582us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:34:24.381435 (+ 503us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:34:24.381441 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:34:24.381446 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:34:24.381594 (+ 148us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:34:24.381872 (+ 278us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:34:24.381876 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:34:24.381877 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:34:24.382397 (+ 520us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:34:24.382591 (+ 194us) server_negotiation.cc:1035] Waiting for connection context
0806 18:34:24.382872 (+ 281us) server_negotiation.cc:299] Negotiation successful
0806 18:34:24.383213 (+ 341us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":246,"thread_start_us":128,"threads_started":1}
I20260806 18:34:24.385048 23454 consensus_queue.cc:237] T 00000000000000000000000000000000 P d321f16d40b14d33aa6b39ec0df23568 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d321f16d40b14d33aa6b39ec0df23568" member_type: VOTER last_known_addr { host: "127.19.194.62" port: 46845 } }
I20260806 18:34:24.385099 23449 sys_catalog.cc:564] T 00000000000000000000000000000000 P d321f16d40b14d33aa6b39ec0df23568 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 18:34:24.387063 23456 sys_catalog.cc:455] T 00000000000000000000000000000000 P d321f16d40b14d33aa6b39ec0df23568 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "d321f16d40b14d33aa6b39ec0df23568" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d321f16d40b14d33aa6b39ec0df23568" member_type: VOTER last_known_addr { host: "127.19.194.62" port: 46845 } } }
I20260806 18:34:24.387219 23456 sys_catalog.cc:458] T 00000000000000000000000000000000 P d321f16d40b14d33aa6b39ec0df23568 [sys.catalog]: This master's current role is: LEADER
I20260806 18:34:24.387548 23455 sys_catalog.cc:455] T 00000000000000000000000000000000 P d321f16d40b14d33aa6b39ec0df23568 [sys.catalog]: SysCatalogTable state changed. Reason: New leader d321f16d40b14d33aa6b39ec0df23568. Latest consensus state: current_term: 1 leader_uuid: "d321f16d40b14d33aa6b39ec0df23568" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d321f16d40b14d33aa6b39ec0df23568" member_type: VOTER last_known_addr { host: "127.19.194.62" port: 46845 } } }
I20260806 18:34:24.387919 23455 sys_catalog.cc:458] T 00000000000000000000000000000000 P d321f16d40b14d33aa6b39ec0df23568 [sys.catalog]: This master's current role is: LEADER
W20260806 18:34:24.390029 23468 catalog_manager.cc:1558] T 00000000000000000000000000000000 P d321f16d40b14d33aa6b39ec0df23568: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260806 18:34:24.390108 23468 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260806 18:34:24.390164 23469 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 18:34:24.390821 23469 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 18:34:24.395915 23469 catalog_manager.cc:1347] Generated new cluster ID: fcc99562606443cdb78443d99c27e023
I20260806 18:34:24.395973 23469 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 18:34:24.415452 23469 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 18:34:24.416678 23469 catalog_manager.cc:1504] Loading token signing keys...
I20260806 18:34:24.421976 23469 catalog_manager.cc:5868] T 00000000000000000000000000000000 P d321f16d40b14d33aa6b39ec0df23568: Generated new TSK 0
I20260806 18:34:24.422716 23469 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.19.194.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.19.194.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.19.194.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786041246094603-20232-0/krb5kdc/kudu_127.19.194.1.keytab.
Entry for principal kudu/127.19.194.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786041246094603-20232-0/krb5kdc/kudu_127.19.194.1.keytab.
I20260806 18:34:24.457675 20232 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJW69tn/build/debug/bin/kudu
/tmp/dist-test-taskJW69tn/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786041246094603-20232-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786041246094603-20232-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786041246094603-20232-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786041246094603-20232-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.19.194.1:0
--local_ip_for_outbound_sockets=127.19.194.1
--webserver_interface=127.19.194.1
--webserver_port=0
--tserver_master_addrs=127.19.194.62:46845
--keytab_file=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786041246094603-20232-0/krb5kdc/kudu_127.19.194.1.keytab
--principal=kudu/127.19.194.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.19.194.20:33261
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786041246094603-20232-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786041246094603-20232-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786041246094603-20232-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 18:34:24.566009 23476 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 18:34:24.566277 23476 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 18:34:24.566349 23476 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 18:34:24.570593 23476 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 18:34:24.570676 23476 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 18:34:24.570828 23476 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.19.194.1
I20260806 18:34:24.576047 23476 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.194.20:33261
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786041246094603-20232-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786041246094603-20232-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786041246094603-20232-0/krb5kdc/kudu_127.19.194.1.keytab
--principal=kudu/127.19.194.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.19.194.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786041246094603-20232-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.19.194.1
--webserver_port=0
--tserver_master_addrs=127.19.194.62:46845
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23476
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.19.194.1
--log_dir=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786041246094603-20232-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 17:59:13 UTC on ccycloud.quasar-wvsnzx.root.comops.site
build id 4571028
I20260806 18:34:24.576999 23476 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 18:34:24.577947 23476 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 18:34:24.587345 23482 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:34:24.587373 23485 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 18:34:24.587617 23476 server_base.cc:1034] running on GCE node
W20260806 18:34:24.587383 23483 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 18:34:24.588057 23476 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 18:34:24.588649 23476 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 18:34:24.589833 23476 hybrid_clock.cc:648] HybridClock initialized: now 1786041264589808 us; error 29 us; skew 500 ppm
Aug 06 18:34:24 dist-test-slave-1xrh krb5kdc[23367](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041264, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.194.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 18:34:24.592602 23476 init.cc:377] Logged in from keytab as kudu/127.19.194.1@KRBTEST.COM (short username kudu)
I20260806 18:34:24.593444 23476 webserver.cc:460] Webserver started at http://127.19.194.1:42479/ using document root <none> and password file <none>
I20260806 18:34:24.593936 23476 fs_manager.cc:359] Metadata directory not provided
I20260806 18:34:24.593993 23476 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 18:34:24.594153 23476 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 18:34:24.596148 23476 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786041246094603-20232-0/minicluster-data/ts-0/data/instance:
uuid: "f68aff679fc14b85b8b330fda3c23061"
format_stamp: "Formatted at 2026-08-06 18:34:24 on dist-test-slave-1xrh"
server_key: "ee610257db2029ac32823daada86504b"
server_key_iv: "23645ec9220f4744fd5247586ece5253"
server_key_version: "encryptionkey@0"
I20260806 18:34:24.596674 23476 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786041246094603-20232-0/minicluster-data/ts-0/wal/instance:
uuid: "f68aff679fc14b85b8b330fda3c23061"
format_stamp: "Formatted at 2026-08-06 18:34:24 on dist-test-slave-1xrh"
server_key: "ee610257db2029ac32823daada86504b"
server_key_iv: "23645ec9220f4744fd5247586ece5253"
server_key_version: "encryptionkey@0"
I20260806 18:34:24.600123 23476 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20260806 18:34:24.602447 23492 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 18:34:24.603686 23476 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260806 18:34:24.603835 23476 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786041246094603-20232-0/minicluster-data/ts-0/data,/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786041246094603-20232-0/minicluster-data/ts-0/wal
uuid: "f68aff679fc14b85b8b330fda3c23061"
format_stamp: "Formatted at 2026-08-06 18:34:24 on dist-test-slave-1xrh"
server_key: "ee610257db2029ac32823daada86504b"
server_key_iv: "23645ec9220f4744fd5247586ece5253"
server_key_version: "encryptionkey@0"
I20260806 18:34:24.603945 23476 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786041246094603-20232-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786041246094603-20232-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786041246094603-20232-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 18:34:24.619678 23476 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 18:34:24.622364 23476 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 18:34:24.622593 23476 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 18:34:24.623204 23476 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 18:34:24.624006 23476 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 18:34:24.624079 23476 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 18:34:24.624150 23476 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 18:34:24.624202 23476 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 18:34:24.633196 23476 rpc_server.cc:307] RPC server started. Bound to: 127.19.194.1:43407
I20260806 18:34:24.633219 23610 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.194.1:43407 every 8 connection(s)
I20260806 18:34:24.633991 23476 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786041246094603-20232-0/minicluster-data/ts-0/data/info.pb
Aug 06 18:34:24 dist-test-slave-1xrh krb5kdc[23367](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041264, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.194.1@KRBTEST.COM for kudu/127.19.194.62@KRBTEST.COM
I20260806 18:34:24.643923 20232 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJW69tn/build/debug/bin/kudu as pid 23476
I20260806 18:34:24.644063 20232 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786041246094603-20232-0/minicluster-data/ts-0/wal/instance
I20260806 18:34:24.644347 20232 external_mini_cluster.cc:1428] Setting key c44b287df10a038618a81780f0ac7a61
I20260806 18:34:24.647334 23452 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:34:24.635775 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.194.1:49443 (local address 127.19.194.62:46845)
0806 18:34:24.635914 (+ 139us) server_negotiation.cc:206] Beginning negotiation
0806 18:34:24.635918 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 18:34:24.636771 (+ 853us) server_negotiation.cc:407] Connection header received
0806 18:34:24.638054 (+ 1283us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:34:24.638057 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:34:24.638115 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:34:24.638188 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:34:24.639265 (+ 1077us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:34:24.639784 (+ 519us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:34:24.640883 (+ 1099us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:34:24.641101 (+ 218us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:34:24.641181 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:34:24.643537 (+ 2356us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:34:24.643540 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:34:24.643542 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 18:34:24.643565 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:34:24.645192 (+ 1627us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:34:24.645761 (+ 569us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:34:24.645765 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:34:24.645767 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:34:24.645885 (+ 118us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:34:24.646236 (+ 351us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:34:24.646240 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:34:24.646242 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:34:24.646424 (+ 182us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:34:24.646541 (+ 117us) server_negotiation.cc:1035] Waiting for connection context
0806 18:34:24.646997 (+ 456us) server_negotiation.cc:299] Negotiation successful
0806 18:34:24.647165 (+ 168us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260806 18:34:24.648214 23613 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:34:24.636131 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.194.62:46845 (local address 127.19.194.1:49443)
0806 18:34:24.636626 (+ 495us) negotiation.cc:107] Waiting for socket to connect
0806 18:34:24.636666 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0806 18:34:24.637857 (+ 1191us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:34:24.638338 (+ 481us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:34:24.638344 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:34:24.638655 (+ 311us) client_negotiation.cc:189] Negotiated authn=SASL
0806 18:34:24.639093 (+ 438us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:34:24.639105 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:34:24.639909 (+ 804us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:34:24.639912 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:34:24.640781 (+ 869us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:34:24.640789 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:34:24.641229 (+ 440us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:34:24.641231 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:34:24.641380 (+ 149us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:34:24.641944 (+ 564us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 18:34:24.641965 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0806 18:34:24.643388 (+ 1423us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 18:34:24.645302 (+ 1914us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:34:24.645307 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:34:24.645320 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:34:24.645650 (+ 330us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:34:24.645980 (+ 330us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:34:24.645984 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:34:24.645986 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:34:24.646134 (+ 148us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:34:24.646543 (+ 409us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 18:34:24.646551 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 18:34:24.646769 (+ 218us) client_negotiation.cc:769] Sending connection context
0806 18:34:24.646952 (+ 183us) client_negotiation.cc:240] Negotiation successful
0806 18:34:24.647292 (+ 340us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":284,"thread_start_us":136,"threads_started":1}
I20260806 18:34:24.649389 23611 heartbeater.cc:347] Connected to a master server at 127.19.194.62:46845
I20260806 18:34:24.649639 23611 heartbeater.cc:464] Registering TS with master...
I20260806 18:34:24.650213 23611 heartbeater.cc:511] Master 127.19.194.62:46845 requested a full tablet report, sending...
I20260806 18:34:24.651731 23413 ts_manager.cc:194] Registered new tserver with Master: f68aff679fc14b85b8b330fda3c23061 (127.19.194.1:43407)
I20260806 18:34:24.652871 23413 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.19.194.1@KRBTEST.COM'} at 127.19.194.1:49443
WARNING: no policy specified for kudu/127.19.194.2@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.19.194.2@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.19.194.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786041246094603-20232-0/krb5kdc/kudu_127.19.194.2.keytab.
Entry for principal kudu/127.19.194.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786041246094603-20232-0/krb5kdc/kudu_127.19.194.2.keytab.
I20260806 18:34:24.672122 20232 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJW69tn/build/debug/bin/kudu
/tmp/dist-test-taskJW69tn/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786041246094603-20232-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786041246094603-20232-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786041246094603-20232-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786041246094603-20232-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.19.194.2:0
--local_ip_for_outbound_sockets=127.19.194.2
--webserver_interface=127.19.194.2
--webserver_port=0
--tserver_master_addrs=127.19.194.62:46845
--keytab_file=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786041246094603-20232-0/krb5kdc/kudu_127.19.194.2.keytab
--principal=kudu/127.19.194.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.19.194.20:33261
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786041246094603-20232-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786041246094603-20232-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786041246094603-20232-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 18:34:24.784395 23617 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 18:34:24.784646 23617 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 18:34:24.784749 23617 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 18:34:24.789482 23617 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 18:34:24.789542 23617 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 18:34:24.789605 23617 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.19.194.2
I20260806 18:34:24.795209 23617 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.194.20:33261
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786041246094603-20232-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786041246094603-20232-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786041246094603-20232-0/krb5kdc/kudu_127.19.194.2.keytab
--principal=kudu/127.19.194.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.19.194.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786041246094603-20232-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.19.194.2
--webserver_port=0
--tserver_master_addrs=127.19.194.62:46845
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23617
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.19.194.2
--log_dir=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786041246094603-20232-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 17:59:13 UTC on ccycloud.quasar-wvsnzx.root.comops.site
build id 4571028
I20260806 18:34:24.796097 23617 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 18:34:24.797044 23617 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 18:34:24.806293 23625 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:34:24.806392 23622 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:34:24.806293 23623 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 18:34:24.806684 23617 server_base.cc:1034] running on GCE node
I20260806 18:34:24.807108 23617 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 18:34:24.807884 23617 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 18:34:24.809072 23617 hybrid_clock.cc:648] HybridClock initialized: now 1786041264809050 us; error 40 us; skew 500 ppm
Aug 06 18:34:24 dist-test-slave-1xrh krb5kdc[23367](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041264, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.194.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 18:34:24.811786 23617 init.cc:377] Logged in from keytab as kudu/127.19.194.2@KRBTEST.COM (short username kudu)
I20260806 18:34:24.812642 23617 webserver.cc:460] Webserver started at http://127.19.194.2:44791/ using document root <none> and password file <none>
I20260806 18:34:24.813174 23617 fs_manager.cc:359] Metadata directory not provided
I20260806 18:34:24.813261 23617 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 18:34:24.813483 23617 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 18:34:24.815531 23617 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786041246094603-20232-0/minicluster-data/ts-1/data/instance:
uuid: "0d805cfb6f34403595c5bcbd9d315204"
format_stamp: "Formatted at 2026-08-06 18:34:24 on dist-test-slave-1xrh"
server_key: "ad138e4f38e6eddac7043d19b2e602a6"
server_key_iv: "72505db11ff9aaf708512e06fa64322b"
server_key_version: "encryptionkey@0"
I20260806 18:34:24.816112 23617 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786041246094603-20232-0/minicluster-data/ts-1/wal/instance:
uuid: "0d805cfb6f34403595c5bcbd9d315204"
format_stamp: "Formatted at 2026-08-06 18:34:24 on dist-test-slave-1xrh"
server_key: "ad138e4f38e6eddac7043d19b2e602a6"
server_key_iv: "72505db11ff9aaf708512e06fa64322b"
server_key_version: "encryptionkey@0"
I20260806 18:34:24.819689 23617 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.001s
I20260806 18:34:24.821991 23632 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 18:34:24.823210 23617 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260806 18:34:24.823328 23617 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786041246094603-20232-0/minicluster-data/ts-1/data,/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786041246094603-20232-0/minicluster-data/ts-1/wal
uuid: "0d805cfb6f34403595c5bcbd9d315204"
format_stamp: "Formatted at 2026-08-06 18:34:24 on dist-test-slave-1xrh"
server_key: "ad138e4f38e6eddac7043d19b2e602a6"
server_key_iv: "72505db11ff9aaf708512e06fa64322b"
server_key_version: "encryptionkey@0"
I20260806 18:34:24.823426 23617 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786041246094603-20232-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786041246094603-20232-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786041246094603-20232-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 18:34:24.840946 23617 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 18:34:24.843837 23617 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 18:34:24.844031 23617 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 18:34:24.844583 23617 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 18:34:24.845366 23617 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 18:34:24.845433 23617 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 18:34:24.845504 23617 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 18:34:24.845552 23617 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 18:34:24.854252 23617 rpc_server.cc:307] RPC server started. Bound to: 127.19.194.2:39473
I20260806 18:34:24.854297 23751 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.194.2:39473 every 8 connection(s)
I20260806 18:34:24.855165 23617 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786041246094603-20232-0/minicluster-data/ts-1/data/info.pb
I20260806 18:34:24.858050 20232 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJW69tn/build/debug/bin/kudu as pid 23617
I20260806 18:34:24.858186 20232 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786041246094603-20232-0/minicluster-data/ts-1/wal/instance
I20260806 18:34:24.858507 20232 external_mini_cluster.cc:1428] Setting key 8739a46512ccc7f0ed2e173398cc288c
Aug 06 18:34:24 dist-test-slave-1xrh krb5kdc[23367](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041264, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.194.2@KRBTEST.COM for kudu/127.19.194.62@KRBTEST.COM
WARNING: no policy specified for kudu/127.19.194.3@KRBTEST.COM; defaulting to no policy
I20260806 18:34:24.869186 23452 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:34:24.856963 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.194.2:55733 (local address 127.19.194.62:46845)
0806 18:34:24.857097 (+ 134us) server_negotiation.cc:206] Beginning negotiation
0806 18:34:24.857100 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 18:34:24.857954 (+ 854us) server_negotiation.cc:407] Connection header received
0806 18:34:24.859600 (+ 1646us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:34:24.859603 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:34:24.859658 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:34:24.859736 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:34:24.860848 (+ 1112us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:34:24.861380 (+ 532us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:34:24.862542 (+ 1162us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:34:24.862781 (+ 239us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:34:24.862896 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:34:24.865647 (+ 2751us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:34:24.865650 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:34:24.865652 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 18:34:24.865675 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:34:24.867091 (+ 1416us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:34:24.867641 (+ 550us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:34:24.867645 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:34:24.867647 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:34:24.867727 (+ 80us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:34:24.868062 (+ 335us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:34:24.868065 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:34:24.868067 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:34:24.868287 (+ 220us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:34:24.868394 (+ 107us) server_negotiation.cc:1035] Waiting for connection context
0806 18:34:24.868857 (+ 463us) server_negotiation.cc:299] Negotiation successful
0806 18:34:24.868990 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260806 18:34:24.869899 23754 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:34:24.857333 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.194.62:46845 (local address 127.19.194.2:55733)
0806 18:34:24.857795 (+ 462us) negotiation.cc:107] Waiting for socket to connect
0806 18:34:24.857838 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0806 18:34:24.859391 (+ 1553us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:34:24.859895 (+ 504us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:34:24.859903 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:34:24.860248 (+ 345us) client_negotiation.cc:189] Negotiated authn=SASL
0806 18:34:24.860693 (+ 445us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:34:24.860706 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:34:24.861556 (+ 850us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:34:24.861560 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:34:24.862413 (+ 853us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:34:24.862422 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:34:24.862934 (+ 512us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:34:24.862938 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:34:24.863130 (+ 192us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:34:24.863772 (+ 642us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 18:34:24.863801 (+ 29us) client_negotiation.cc:656] calling sasl_client_start()
0806 18:34:24.865447 (+ 1646us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 18:34:24.867247 (+ 1800us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:34:24.867253 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:34:24.867267 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:34:24.867530 (+ 263us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:34:24.867819 (+ 289us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:34:24.867823 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:34:24.867825 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:34:24.867964 (+ 139us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:34:24.868393 (+ 429us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 18:34:24.868400 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 18:34:24.868630 (+ 230us) client_negotiation.cc:769] Sending connection context
0806 18:34:24.868812 (+ 182us) client_negotiation.cc:240] Negotiation successful
0806 18:34:24.869116 (+ 304us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":283,"thread_start_us":122,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.19.194.3@KRBTEST.COM" created.
I20260806 18:34:24.871165 23752 heartbeater.cc:347] Connected to a master server at 127.19.194.62:46845
I20260806 18:34:24.871443 23752 heartbeater.cc:464] Registering TS with master...
I20260806 18:34:24.872037 23752 heartbeater.cc:511] Master 127.19.194.62:46845 requested a full tablet report, sending...
I20260806 18:34:24.873139 23413 ts_manager.cc:194] Registered new tserver with Master: 0d805cfb6f34403595c5bcbd9d315204 (127.19.194.2:39473)
I20260806 18:34:24.873692 23413 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.19.194.2@KRBTEST.COM'} at 127.19.194.2:55733
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.19.194.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786041246094603-20232-0/krb5kdc/kudu_127.19.194.3.keytab.
Entry for principal kudu/127.19.194.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786041246094603-20232-0/krb5kdc/kudu_127.19.194.3.keytab.
I20260806 18:34:24.886853 20232 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJW69tn/build/debug/bin/kudu
/tmp/dist-test-taskJW69tn/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786041246094603-20232-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786041246094603-20232-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786041246094603-20232-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786041246094603-20232-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.19.194.3:0
--local_ip_for_outbound_sockets=127.19.194.3
--webserver_interface=127.19.194.3
--webserver_port=0
--tserver_master_addrs=127.19.194.62:46845
--keytab_file=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786041246094603-20232-0/krb5kdc/kudu_127.19.194.3.keytab
--principal=kudu/127.19.194.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.19.194.20:33261
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786041246094603-20232-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786041246094603-20232-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786041246094603-20232-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 18:34:24.996856 23757 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 18:34:24.997115 23757 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 18:34:24.997159 23757 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 18:34:25.001106 23757 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 18:34:25.001157 23757 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 18:34:25.001214 23757 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.19.194.3
I20260806 18:34:25.006549 23757 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.194.20:33261
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786041246094603-20232-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786041246094603-20232-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786041246094603-20232-0/krb5kdc/kudu_127.19.194.3.keytab
--principal=kudu/127.19.194.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.19.194.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786041246094603-20232-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.19.194.3
--webserver_port=0
--tserver_master_addrs=127.19.194.62:46845
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23757
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.19.194.3
--log_dir=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786041246094603-20232-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 17:59:13 UTC on ccycloud.quasar-wvsnzx.root.comops.site
build id 4571028
I20260806 18:34:25.007587 23757 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 18:34:25.008589 23757 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 18:34:25.018249 23762 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:34:25.018362 23766 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:34:25.018282 23763 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 18:34:25.018560 23757 server_base.cc:1034] running on GCE node
I20260806 18:34:25.018942 23757 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 18:34:25.019624 23757 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 18:34:25.020807 23757 hybrid_clock.cc:648] HybridClock initialized: now 1786041265020793 us; error 26 us; skew 500 ppm
Aug 06 18:34:25 dist-test-slave-1xrh krb5kdc[23367](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041265, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.194.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 18:34:25.023485 23757 init.cc:377] Logged in from keytab as kudu/127.19.194.3@KRBTEST.COM (short username kudu)
I20260806 18:34:25.024271 23757 webserver.cc:460] Webserver started at http://127.19.194.3:45183/ using document root <none> and password file <none>
I20260806 18:34:25.024772 23757 fs_manager.cc:359] Metadata directory not provided
I20260806 18:34:25.024829 23757 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 18:34:25.024987 23757 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 18:34:25.026950 23757 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786041246094603-20232-0/minicluster-data/ts-2/data/instance:
uuid: "6bb50bc9fa1a4da8a728490d576b55a7"
format_stamp: "Formatted at 2026-08-06 18:34:25 on dist-test-slave-1xrh"
server_key: "0e6e934370b613487f519ca5803ff0ff"
server_key_iv: "530e57c652eb3d2d6340002463899df5"
server_key_version: "encryptionkey@0"
I20260806 18:34:25.027581 23757 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786041246094603-20232-0/minicluster-data/ts-2/wal/instance:
uuid: "6bb50bc9fa1a4da8a728490d576b55a7"
format_stamp: "Formatted at 2026-08-06 18:34:25 on dist-test-slave-1xrh"
server_key: "0e6e934370b613487f519ca5803ff0ff"
server_key_iv: "530e57c652eb3d2d6340002463899df5"
server_key_version: "encryptionkey@0"
I20260806 18:34:25.031046 23757 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.000s
I20260806 18:34:25.033393 23773 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 18:34:25.034632 23757 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260806 18:34:25.034771 23757 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786041246094603-20232-0/minicluster-data/ts-2/data,/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786041246094603-20232-0/minicluster-data/ts-2/wal
uuid: "6bb50bc9fa1a4da8a728490d576b55a7"
format_stamp: "Formatted at 2026-08-06 18:34:25 on dist-test-slave-1xrh"
server_key: "0e6e934370b613487f519ca5803ff0ff"
server_key_iv: "530e57c652eb3d2d6340002463899df5"
server_key_version: "encryptionkey@0"
I20260806 18:34:25.034880 23757 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786041246094603-20232-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786041246094603-20232-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786041246094603-20232-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 18:34:25.046109 23757 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 18:34:25.049250 23757 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 18:34:25.049444 23757 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 18:34:25.049985 23757 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 18:34:25.050767 23757 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 18:34:25.050853 23757 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 18:34:25.050920 23757 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 18:34:25.050966 23757 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 18:34:25.059773 23757 rpc_server.cc:307] RPC server started. Bound to: 127.19.194.3:40909
I20260806 18:34:25.059811 23892 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.194.3:40909 every 8 connection(s)
I20260806 18:34:25.060599 23757 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786041246094603-20232-0/minicluster-data/ts-2/data/info.pb
I20260806 18:34:25.063263 20232 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJW69tn/build/debug/bin/kudu as pid 23757
I20260806 18:34:25.063392 20232 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786041246094603-20232-0/minicluster-data/ts-2/wal/instance
I20260806 18:34:25.063676 20232 external_mini_cluster.cc:1428] Setting key 2444b9695a9c3962557bb68faa15dad5
Aug 06 18:34:25 dist-test-slave-1xrh krb5kdc[23367](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041265, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.194.3@KRBTEST.COM for kudu/127.19.194.62@KRBTEST.COM
I20260806 18:34:25.074147 23452 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:34:25.062254 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.194.3:52313 (local address 127.19.194.62:46845)
0806 18:34:25.062386 (+ 132us) server_negotiation.cc:206] Beginning negotiation
0806 18:34:25.062390 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 18:34:25.063110 (+ 720us) server_negotiation.cc:407] Connection header received
0806 18:34:25.064800 (+ 1690us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:34:25.064803 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:34:25.064861 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:34:25.064974 (+ 113us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:34:25.066277 (+ 1303us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:34:25.066810 (+ 533us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:34:25.068000 (+ 1190us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:34:25.068224 (+ 224us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:34:25.068354 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:34:25.070722 (+ 2368us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:34:25.070725 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:34:25.070727 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 18:34:25.070748 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:34:25.072136 (+ 1388us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:34:25.072659 (+ 523us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:34:25.072662 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:34:25.072664 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:34:25.072746 (+ 82us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:34:25.073084 (+ 338us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:34:25.073087 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:34:25.073089 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:34:25.073265 (+ 176us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:34:25.073375 (+ 110us) server_negotiation.cc:1035] Waiting for connection context
0806 18:34:25.073838 (+ 463us) server_negotiation.cc:299] Negotiation successful
0806 18:34:25.073954 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260806 18:34:25.074856 23895 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:34:25.062539 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.194.62:46845 (local address 127.19.194.3:52313)
0806 18:34:25.062966 (+ 427us) negotiation.cc:107] Waiting for socket to connect
0806 18:34:25.063007 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0806 18:34:25.064571 (+ 1564us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:34:25.065162 (+ 591us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:34:25.065171 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:34:25.065648 (+ 477us) client_negotiation.cc:189] Negotiated authn=SASL
0806 18:34:25.066133 (+ 485us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:34:25.066145 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:34:25.066938 (+ 793us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:34:25.066942 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:34:25.067862 (+ 920us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:34:25.067872 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:34:25.068392 (+ 520us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:34:25.068396 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:34:25.068563 (+ 167us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:34:25.069049 (+ 486us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 18:34:25.069064 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0806 18:34:25.070554 (+ 1490us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 18:34:25.072281 (+ 1727us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:34:25.072287 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:34:25.072300 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:34:25.072553 (+ 253us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:34:25.072840 (+ 287us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:34:25.072844 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:34:25.072846 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:34:25.072983 (+ 137us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:34:25.073374 (+ 391us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 18:34:25.073381 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 18:34:25.073614 (+ 233us) client_negotiation.cc:769] Sending connection context
0806 18:34:25.073794 (+ 180us) client_negotiation.cc:240] Negotiation successful
0806 18:34:25.074103 (+ 309us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":244,"thread_start_us":105,"threads_started":1}
I20260806 18:34:25.075887 23893 heartbeater.cc:347] Connected to a master server at 127.19.194.62:46845
I20260806 18:34:25.076092 23893 heartbeater.cc:464] Registering TS with master...
I20260806 18:34:25.076603 23893 heartbeater.cc:511] Master 127.19.194.62:46845 requested a full tablet report, sending...
I20260806 18:34:25.077553 23413 ts_manager.cc:194] Registered new tserver with Master: 6bb50bc9fa1a4da8a728490d576b55a7 (127.19.194.3:40909)
I20260806 18:34:25.078119 23413 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.19.194.3@KRBTEST.COM'} at 127.19.194.3:52313
I20260806 18:34:25.088930 20232 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 06 18:34:25 dist-test-slave-1xrh krb5kdc[23367](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041265, etypes {rep=17 tkt=17 ses=17}, joe-interloper@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for joe-interloper@KRBTEST.COM:
Aug 06 18:34:25 dist-test-slave-1xrh krb5kdc[23367](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041265, etypes {rep=17 tkt=17 ses=17}, joe-interloper@KRBTEST.COM for kudu/127.19.194.62@KRBTEST.COM
I20260806 18:34:25.114285 23452 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:34:25.105821 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40708 (local address 127.19.194.62:46845)
0806 18:34:25.105983 (+ 162us) server_negotiation.cc:206] Beginning negotiation
0806 18:34:25.105988 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 18:34:25.106238 (+ 250us) server_negotiation.cc:407] Connection header received
0806 18:34:25.106398 (+ 160us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:34:25.106407 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:34:25.106448 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:34:25.106534 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:34:25.107266 (+ 732us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:34:25.107782 (+ 516us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:34:25.108703 (+ 921us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:34:25.108905 (+ 202us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:34:25.109036 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:34:25.111388 (+ 2352us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:34:25.111391 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:34:25.111393 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 18:34:25.111414 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:34:25.112577 (+ 1163us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:34:25.112995 (+ 418us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:34:25.112999 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:34:25.113002 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:34:25.113080 (+ 78us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:34:25.113360 (+ 280us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:34:25.113363 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:34:25.113365 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:34:25.113578 (+ 213us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:34:25.113721 (+ 143us) server_negotiation.cc:1035] Waiting for connection context
0806 18:34:25.113987 (+ 266us) server_negotiation.cc:299] Negotiation successful
0806 18:34:25.114102 (+ 115us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":92}
W20260806 18:34:25.114661 23413 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.ConnectToMaster from {username='joe-interloper', principal='joe-interloper@KRBTEST.COM'} at 127.0.0.1:40708
I20260806 18:34:25.115559 20232 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJW69tn/build/debug/bin/kudu with pid 23476
I20260806 18:34:25.123415 20232 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJW69tn/build/debug/bin/kudu with pid 23617
I20260806 18:34:25.130450 20232 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJW69tn/build/debug/bin/kudu with pid 23757
I20260806 18:34:25.136649 20232 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJW69tn/build/debug/bin/kudu with pid 23380
2026-08-06T18:34:25Z chronyd exiting
[ OK ] SecurityITest.TestUnauthorizedClientKerberosCredentials (2123 ms)
[ RUN ] SecurityITest.TestAuthorizedSuperuser
Loading random data
Initializing database '/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786041246094603-20232-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 18:34:25 dist-test-slave-1xrh krb5kdc[23905](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 18:34:25 dist-test-slave-1xrh krb5kdc[23905](info): set up 2 sockets
Aug 06 18:34:25 dist-test-slave-1xrh krb5kdc[23905](info): commencing operation
krb5kdc: starting...
W20260806 18:34:26.334987 20232 mini_kdc.cc:121] Time spent starting KDC: real 1.175s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
W20260806 18:34:26.494614 20232 mini_kdc.cc:253] Time spent creating user principal test-admin: real 0.159s user 0.000s sys 0.002s
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
W20260806 18:34:26.619817 20232 mini_kdc.cc:253] Time spent creating user principal test-user: real 0.125s user 0.000s sys 0.002s
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
W20260806 18:34:26.769696 20232 mini_kdc.cc:253] Time spent creating user principal joe-interloper: real 0.150s user 0.000s sys 0.002s
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786041246094603-20232-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786041246094603-20232-0/krb5kdc/test-user.keytab.
W20260806 18:34:27.010844 20232 mini_kdc.cc:302] Time spent creating keytab for test-user: real 0.241s user 0.000s sys 0.002s
Aug 06 18:34:27 dist-test-slave-1xrh krb5kdc[23905](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041267, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T18:34:27Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T18:34:27Z Disabled control of system clock
WARNING: no policy specified for kudu/127.19.194.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.19.194.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.19.194.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786041246094603-20232-0/krb5kdc/kudu_127.19.194.62.keytab.
Entry for principal kudu/127.19.194.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786041246094603-20232-0/krb5kdc/kudu_127.19.194.62.keytab.
I20260806 18:34:27.057948 20232 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJW69tn/build/debug/bin/kudu
/tmp/dist-test-taskJW69tn/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786041246094603-20232-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786041246094603-20232-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786041246094603-20232-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786041246094603-20232-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.19.194.62:44971
--webserver_interface=127.19.194.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786041246094603-20232-0/krb5kdc/kudu_127.19.194.62.keytab
--principal=kudu/127.19.194.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.19.194.20:34191
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.19.194.62:44971
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786041246094603-20232-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786041246094603-20232-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786041246094603-20232-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 18:34:27.165774 23918 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 18:34:27.166025 23918 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 18:34:27.166064 23918 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 18:34:27.171350 23918 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 18:34:27.171411 23918 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 18:34:27.171430 23918 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 18:34:27.171447 23918 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 18:34:27.171468 23918 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 18:34:27.176949 23918 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.194.20:34191
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786041246094603-20232-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786041246094603-20232-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.19.194.62:44971
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786041246094603-20232-0/krb5kdc/kudu_127.19.194.62.keytab
--principal=kudu/127.19.194.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.19.194.62:44971
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786041246094603-20232-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.19.194.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23918
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786041246094603-20232-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 17:59:13 UTC on ccycloud.quasar-wvsnzx.root.comops.site
build id 4571028
I20260806 18:34:27.177911 23918 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 18:34:27.178865 23918 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 18:34:27.187220 23926 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:34:27.187232 23928 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 18:34:27.187371 23918 server_base.cc:1034] running on GCE node
W20260806 18:34:27.187225 23925 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 18:34:27.188122 23918 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 18:34:27.189227 23918 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 18:34:27.190390 23918 hybrid_clock.cc:648] HybridClock initialized: now 1786041267190373 us; error 33 us; skew 500 ppm
Aug 06 18:34:27 dist-test-slave-1xrh krb5kdc[23905](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041267, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.194.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 18:34:27.193274 23918 init.cc:377] Logged in from keytab as kudu/127.19.194.62@KRBTEST.COM (short username kudu)
I20260806 18:34:27.194116 23918 webserver.cc:460] Webserver started at http://127.19.194.62:33491/ using document root <none> and password file <none>
I20260806 18:34:27.194657 23918 fs_manager.cc:359] Metadata directory not provided
I20260806 18:34:27.194747 23918 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 18:34:27.195005 23918 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 18:34:27.197074 23918 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786041246094603-20232-0/minicluster-data/master-0/data/instance:
uuid: "73c59e54d4114b8b829faaa500a73791"
format_stamp: "Formatted at 2026-08-06 18:34:27 on dist-test-slave-1xrh"
server_key: "7272b95b430b080be001996a3f725177"
server_key_iv: "074e11e0e4d0f7bc0af9a2046fdd2f64"
server_key_version: "encryptionkey@0"
I20260806 18:34:27.197625 23918 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786041246094603-20232-0/minicluster-data/master-0/wal/instance:
uuid: "73c59e54d4114b8b829faaa500a73791"
format_stamp: "Formatted at 2026-08-06 18:34:27 on dist-test-slave-1xrh"
server_key: "7272b95b430b080be001996a3f725177"
server_key_iv: "074e11e0e4d0f7bc0af9a2046fdd2f64"
server_key_version: "encryptionkey@0"
I20260806 18:34:27.201263 23918 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.005s sys 0.000s
I20260806 18:34:27.203610 23935 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 18:34:27.204813 23918 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260806 18:34:27.204958 23918 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786041246094603-20232-0/minicluster-data/master-0/data,/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786041246094603-20232-0/minicluster-data/master-0/wal
uuid: "73c59e54d4114b8b829faaa500a73791"
format_stamp: "Formatted at 2026-08-06 18:34:27 on dist-test-slave-1xrh"
server_key: "7272b95b430b080be001996a3f725177"
server_key_iv: "074e11e0e4d0f7bc0af9a2046fdd2f64"
server_key_version: "encryptionkey@0"
I20260806 18:34:27.205076 23918 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786041246094603-20232-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786041246094603-20232-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786041246094603-20232-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 18:34:27.219617 23918 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 18:34:27.222254 23918 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 18:34:27.222465 23918 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 18:34:27.229542 23918 rpc_server.cc:307] RPC server started. Bound to: 127.19.194.62:44971
I20260806 18:34:27.229578 23988 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.194.62:44971 every 8 connection(s)
I20260806 18:34:27.230485 23918 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786041246094603-20232-0/minicluster-data/master-0/data/info.pb
I20260806 18:34:27.233080 23989 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 18:34:27.233547 20232 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJW69tn/build/debug/bin/kudu as pid 23918
I20260806 18:34:27.233672 20232 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786041246094603-20232-0/minicluster-data/master-0/wal/instance
I20260806 18:34:27.233944 20232 external_mini_cluster.cc:1428] Setting key 5858937169212221ca2bb34015587b5d
I20260806 18:34:27.239586 23989 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 73c59e54d4114b8b829faaa500a73791: Bootstrap starting.
I20260806 18:34:27.241914 23989 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 73c59e54d4114b8b829faaa500a73791: Neither blocks nor log segments found. Creating new log.
Aug 06 18:34:27 dist-test-slave-1xrh krb5kdc[23905](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041267, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.19.194.62@KRBTEST.COM
I20260806 18:34:27.242736 23989 log.cc:824] T 00000000000000000000000000000000 P 73c59e54d4114b8b829faaa500a73791: Log is configured to *not* fsync() on all Append() calls
I20260806 18:34:27.245097 23989 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 73c59e54d4114b8b829faaa500a73791: No bootstrap required, opened a new log
I20260806 18:34:27.248271 23992 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:34:27.235333 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44808 (local address 127.19.194.62:44971)
0806 18:34:27.235748 (+ 415us) server_negotiation.cc:206] Beginning negotiation
0806 18:34:27.235762 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0806 18:34:27.235796 (+ 34us) server_negotiation.cc:407] Connection header received
0806 18:34:27.236478 (+ 682us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:34:27.236494 (+ 16us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:34:27.236809 (+ 315us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:34:27.237151 (+ 342us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:34:27.238059 (+ 908us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:34:27.239040 (+ 981us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:34:27.239957 (+ 917us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:34:27.240415 (+ 458us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:34:27.240593 (+ 178us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:34:27.243086 (+ 2493us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:34:27.243091 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:34:27.243104 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 18:34:27.243173 (+ 69us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:34:27.245120 (+ 1947us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:34:27.245583 (+ 463us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:34:27.245589 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:34:27.245594 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:34:27.245733 (+ 139us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:34:27.246018 (+ 285us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:34:27.246021 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:34:27.246022 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:34:27.246490 (+ 468us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:34:27.246682 (+ 192us) server_negotiation.cc:1035] Waiting for connection context
0806 18:34:27.246983 (+ 301us) server_negotiation.cc:299] Negotiation successful
0806 18:34:27.247345 (+ 362us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":283,"thread_start_us":131,"threads_started":1}
I20260806 18:34:27.249356 23989 raft_consensus.cc:348] T 00000000000000000000000000000000 P 73c59e54d4114b8b829faaa500a73791 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "73c59e54d4114b8b829faaa500a73791" member_type: VOTER last_known_addr { host: "127.19.194.62" port: 44971 } }
I20260806 18:34:27.249588 23989 raft_consensus.cc:374] T 00000000000000000000000000000000 P 73c59e54d4114b8b829faaa500a73791 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 18:34:27.249646 23989 raft_consensus.cc:729] T 00000000000000000000000000000000 P 73c59e54d4114b8b829faaa500a73791 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 73c59e54d4114b8b829faaa500a73791, State: Initialized, Role: FOLLOWER
I20260806 18:34:27.250079 23989 consensus_queue.cc:260] T 00000000000000000000000000000000 P 73c59e54d4114b8b829faaa500a73791 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "73c59e54d4114b8b829faaa500a73791" member_type: VOTER last_known_addr { host: "127.19.194.62" port: 44971 } }
I20260806 18:34:27.250226 23989 raft_consensus.cc:388] T 00000000000000000000000000000000 P 73c59e54d4114b8b829faaa500a73791 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 18:34:27.250308 23989 raft_consensus.cc:482] T 00000000000000000000000000000000 P 73c59e54d4114b8b829faaa500a73791 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 18:34:27.250420 23989 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 73c59e54d4114b8b829faaa500a73791 [term 0 FOLLOWER]: Advancing to term 1
I20260806 18:34:27.251351 23989 raft_consensus.cc:504] T 00000000000000000000000000000000 P 73c59e54d4114b8b829faaa500a73791 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "73c59e54d4114b8b829faaa500a73791" member_type: VOTER last_known_addr { host: "127.19.194.62" port: 44971 } }
I20260806 18:34:27.251623 23989 leader_election.cc:302] T 00000000000000000000000000000000 P 73c59e54d4114b8b829faaa500a73791 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 73c59e54d4114b8b829faaa500a73791; no voters:
I20260806 18:34:27.251863 23989 leader_election.cc:288] T 00000000000000000000000000000000 P 73c59e54d4114b8b829faaa500a73791 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 18:34:27.251988 23994 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 73c59e54d4114b8b829faaa500a73791 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 18:34:27.252255 23994 raft_consensus.cc:686] T 00000000000000000000000000000000 P 73c59e54d4114b8b829faaa500a73791 [term 1 LEADER]: Becoming Leader. State: Replica: 73c59e54d4114b8b829faaa500a73791, State: Running, Role: LEADER
I20260806 18:34:27.252635 23994 consensus_queue.cc:237] T 00000000000000000000000000000000 P 73c59e54d4114b8b829faaa500a73791 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "73c59e54d4114b8b829faaa500a73791" member_type: VOTER last_known_addr { host: "127.19.194.62" port: 44971 } }
I20260806 18:34:27.253062 23989 sys_catalog.cc:564] T 00000000000000000000000000000000 P 73c59e54d4114b8b829faaa500a73791 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 18:34:27.254347 23997 sys_catalog.cc:455] T 00000000000000000000000000000000 P 73c59e54d4114b8b829faaa500a73791 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 73c59e54d4114b8b829faaa500a73791. Latest consensus state: current_term: 1 leader_uuid: "73c59e54d4114b8b829faaa500a73791" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "73c59e54d4114b8b829faaa500a73791" member_type: VOTER last_known_addr { host: "127.19.194.62" port: 44971 } } }
I20260806 18:34:27.254324 23995 sys_catalog.cc:455] T 00000000000000000000000000000000 P 73c59e54d4114b8b829faaa500a73791 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "73c59e54d4114b8b829faaa500a73791" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "73c59e54d4114b8b829faaa500a73791" member_type: VOTER last_known_addr { host: "127.19.194.62" port: 44971 } } }
I20260806 18:34:27.254516 23997 sys_catalog.cc:458] T 00000000000000000000000000000000 P 73c59e54d4114b8b829faaa500a73791 [sys.catalog]: This master's current role is: LEADER
I20260806 18:34:27.254518 23995 sys_catalog.cc:458] T 00000000000000000000000000000000 P 73c59e54d4114b8b829faaa500a73791 [sys.catalog]: This master's current role is: LEADER
I20260806 18:34:27.255642 24007 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 18:34:27.258538 24007 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 18:34:27.264179 24007 catalog_manager.cc:1347] Generated new cluster ID: dacc4e6a58d6461aadfd51441d9ade4c
I20260806 18:34:27.264247 24007 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 18:34:27.281095 24007 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 18:34:27.281914 24007 catalog_manager.cc:1504] Loading token signing keys...
I20260806 18:34:27.288213 24007 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 73c59e54d4114b8b829faaa500a73791: Generated new TSK 0
I20260806 18:34:27.288834 24007 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.19.194.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.19.194.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.19.194.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786041246094603-20232-0/krb5kdc/kudu_127.19.194.1.keytab.
Entry for principal kudu/127.19.194.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786041246094603-20232-0/krb5kdc/kudu_127.19.194.1.keytab.
I20260806 18:34:27.322377 20232 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJW69tn/build/debug/bin/kudu
/tmp/dist-test-taskJW69tn/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786041246094603-20232-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786041246094603-20232-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786041246094603-20232-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786041246094603-20232-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.19.194.1:0
--local_ip_for_outbound_sockets=127.19.194.1
--webserver_interface=127.19.194.1
--webserver_port=0
--tserver_master_addrs=127.19.194.62:44971
--keytab_file=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786041246094603-20232-0/krb5kdc/kudu_127.19.194.1.keytab
--principal=kudu/127.19.194.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.19.194.20:34191
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786041246094603-20232-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786041246094603-20232-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786041246094603-20232-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 18:34:27.448571 24018 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 18:34:27.448834 24018 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 18:34:27.448910 24018 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 18:34:27.452782 24018 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 18:34:27.452853 24018 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 18:34:27.452950 24018 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.19.194.1
I20260806 18:34:27.458242 24018 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.194.20:34191
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786041246094603-20232-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786041246094603-20232-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786041246094603-20232-0/krb5kdc/kudu_127.19.194.1.keytab
--principal=kudu/127.19.194.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.19.194.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786041246094603-20232-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.19.194.1
--webserver_port=0
--tserver_master_addrs=127.19.194.62:44971
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24018
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.19.194.1
--log_dir=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786041246094603-20232-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 17:59:13 UTC on ccycloud.quasar-wvsnzx.root.comops.site
build id 4571028
I20260806 18:34:27.459230 24018 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 18:34:27.460287 24018 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 18:34:27.469774 24026 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:34:27.469774 24023 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:34:27.469811 24024 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 18:34:27.469883 24018 server_base.cc:1034] running on GCE node
I20260806 18:34:27.470494 24018 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 18:34:27.471236 24018 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 18:34:27.472426 24018 hybrid_clock.cc:648] HybridClock initialized: now 1786041267472399 us; error 43 us; skew 500 ppm
Aug 06 18:34:27 dist-test-slave-1xrh krb5kdc[23905](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041267, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.194.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 18:34:27.475656 24018 init.cc:377] Logged in from keytab as kudu/127.19.194.1@KRBTEST.COM (short username kudu)
I20260806 18:34:27.476516 24018 webserver.cc:460] Webserver started at http://127.19.194.1:38549/ using document root <none> and password file <none>
I20260806 18:34:27.477065 24018 fs_manager.cc:359] Metadata directory not provided
I20260806 18:34:27.477159 24018 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 18:34:27.477383 24018 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 18:34:27.479499 24018 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786041246094603-20232-0/minicluster-data/ts-0/data/instance:
uuid: "10295d0b2b5b462f97aa7572c7638a3e"
format_stamp: "Formatted at 2026-08-06 18:34:27 on dist-test-slave-1xrh"
server_key: "75be88fd95b2920c2d55fa7ebdfb3452"
server_key_iv: "dd75471396293fa31a58babcaa5107a0"
server_key_version: "encryptionkey@0"
I20260806 18:34:27.480088 24018 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786041246094603-20232-0/minicluster-data/ts-0/wal/instance:
uuid: "10295d0b2b5b462f97aa7572c7638a3e"
format_stamp: "Formatted at 2026-08-06 18:34:27 on dist-test-slave-1xrh"
server_key: "75be88fd95b2920c2d55fa7ebdfb3452"
server_key_iv: "dd75471396293fa31a58babcaa5107a0"
server_key_version: "encryptionkey@0"
I20260806 18:34:27.483665 24018 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.001s
I20260806 18:34:27.485921 24034 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 18:34:27.487039 24018 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260806 18:34:27.487192 24018 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786041246094603-20232-0/minicluster-data/ts-0/data,/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786041246094603-20232-0/minicluster-data/ts-0/wal
uuid: "10295d0b2b5b462f97aa7572c7638a3e"
format_stamp: "Formatted at 2026-08-06 18:34:27 on dist-test-slave-1xrh"
server_key: "75be88fd95b2920c2d55fa7ebdfb3452"
server_key_iv: "dd75471396293fa31a58babcaa5107a0"
server_key_version: "encryptionkey@0"
I20260806 18:34:27.487301 24018 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786041246094603-20232-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786041246094603-20232-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786041246094603-20232-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 18:34:27.508172 24018 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 18:34:27.511620 24018 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 18:34:27.511830 24018 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 18:34:27.512408 24018 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 18:34:27.513231 24018 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 18:34:27.513301 24018 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 18:34:27.513366 24018 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 18:34:27.513396 24018 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 18:34:27.522182 24018 rpc_server.cc:307] RPC server started. Bound to: 127.19.194.1:42717
I20260806 18:34:27.522231 24152 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.194.1:42717 every 8 connection(s)
I20260806 18:34:27.523048 24018 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786041246094603-20232-0/minicluster-data/ts-0/data/info.pb
I20260806 18:34:27.529079 20232 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJW69tn/build/debug/bin/kudu as pid 24018
I20260806 18:34:27.529211 20232 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786041246094603-20232-0/minicluster-data/ts-0/wal/instance
I20260806 18:34:27.529522 20232 external_mini_cluster.cc:1428] Setting key 5f94a2d7bf98b826077fd05497d11e78
Aug 06 18:34:27 dist-test-slave-1xrh krb5kdc[23905](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041267, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.194.1@KRBTEST.COM for kudu/127.19.194.62@KRBTEST.COM
I20260806 18:34:27.537055 23992 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:34:27.524809 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.194.1:50191 (local address 127.19.194.62:44971)
0806 18:34:27.524947 (+ 138us) server_negotiation.cc:206] Beginning negotiation
0806 18:34:27.524950 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 18:34:27.525671 (+ 721us) server_negotiation.cc:407] Connection header received
0806 18:34:27.527029 (+ 1358us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:34:27.527034 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:34:27.527109 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:34:27.527243 (+ 134us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:34:27.528317 (+ 1074us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:34:27.528848 (+ 531us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:34:27.530361 (+ 1513us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:34:27.530654 (+ 293us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:34:27.530759 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:34:27.533230 (+ 2471us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:34:27.533236 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:34:27.533240 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 18:34:27.533274 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:34:27.534734 (+ 1460us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:34:27.535404 (+ 670us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:34:27.535406 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:34:27.535407 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:34:27.535496 (+ 89us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:34:27.535903 (+ 407us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:34:27.535906 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:34:27.535907 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:34:27.536078 (+ 171us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:34:27.536156 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0806 18:34:27.536832 (+ 676us) server_negotiation.cc:299] Negotiation successful
0806 18:34:27.536931 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260806 18:34:27.538264 24155 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:34:27.525082 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.194.62:44971 (local address 127.19.194.1:50191)
0806 18:34:27.525514 (+ 432us) negotiation.cc:107] Waiting for socket to connect
0806 18:34:27.525558 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0806 18:34:27.526816 (+ 1258us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:34:27.527405 (+ 589us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:34:27.527411 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:34:27.527730 (+ 319us) client_negotiation.cc:189] Negotiated authn=SASL
0806 18:34:27.528169 (+ 439us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:34:27.528182 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:34:27.528979 (+ 797us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:34:27.528987 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:34:27.530195 (+ 1208us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:34:27.530207 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:34:27.530810 (+ 603us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:34:27.530814 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:34:27.530974 (+ 160us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:34:27.531609 (+ 635us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 18:34:27.531625 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0806 18:34:27.533057 (+ 1432us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 18:34:27.534875 (+ 1818us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:34:27.534884 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:34:27.534901 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:34:27.535303 (+ 402us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:34:27.535597 (+ 294us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:34:27.535602 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:34:27.535605 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:34:27.535804 (+ 199us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:34:27.536193 (+ 389us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 18:34:27.536199 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 18:34:27.536570 (+ 371us) client_negotiation.cc:769] Sending connection context
0806 18:34:27.536804 (+ 234us) client_negotiation.cc:240] Negotiation successful
0806 18:34:27.537165 (+ 361us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":259,"thread_start_us":112,"threads_started":1}
I20260806 18:34:27.539475 24153 heartbeater.cc:347] Connected to a master server at 127.19.194.62:44971
I20260806 18:34:27.539746 24153 heartbeater.cc:464] Registering TS with master...
WARNING: no policy specified for kudu/127.19.194.2@KRBTEST.COM; defaulting to no policy
I20260806 18:34:27.540337 24153 heartbeater.cc:511] Master 127.19.194.62:44971 requested a full tablet report, sending...
I20260806 18:34:27.541837 23953 ts_manager.cc:194] Registered new tserver with Master: 10295d0b2b5b462f97aa7572c7638a3e (127.19.194.1:42717)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.19.194.2@KRBTEST.COM" created.
I20260806 18:34:27.543030 23953 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.19.194.1@KRBTEST.COM'} at 127.19.194.1:50191
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.19.194.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786041246094603-20232-0/krb5kdc/kudu_127.19.194.2.keytab.
Entry for principal kudu/127.19.194.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786041246094603-20232-0/krb5kdc/kudu_127.19.194.2.keytab.
I20260806 18:34:27.558640 20232 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJW69tn/build/debug/bin/kudu
/tmp/dist-test-taskJW69tn/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786041246094603-20232-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786041246094603-20232-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786041246094603-20232-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786041246094603-20232-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.19.194.2:0
--local_ip_for_outbound_sockets=127.19.194.2
--webserver_interface=127.19.194.2
--webserver_port=0
--tserver_master_addrs=127.19.194.62:44971
--keytab_file=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786041246094603-20232-0/krb5kdc/kudu_127.19.194.2.keytab
--principal=kudu/127.19.194.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.19.194.20:34191
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786041246094603-20232-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786041246094603-20232-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786041246094603-20232-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 18:34:27.665727 24158 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 18:34:27.666055 24158 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 18:34:27.666117 24158 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 18:34:27.670111 24158 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 18:34:27.670161 24158 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 18:34:27.670217 24158 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.19.194.2
I20260806 18:34:27.675166 24158 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.194.20:34191
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786041246094603-20232-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786041246094603-20232-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786041246094603-20232-0/krb5kdc/kudu_127.19.194.2.keytab
--principal=kudu/127.19.194.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.19.194.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786041246094603-20232-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.19.194.2
--webserver_port=0
--tserver_master_addrs=127.19.194.62:44971
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24158
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.19.194.2
--log_dir=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786041246094603-20232-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 17:59:13 UTC on ccycloud.quasar-wvsnzx.root.comops.site
build id 4571028
I20260806 18:34:27.676170 24158 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 18:34:27.677167 24158 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 18:34:27.686518 24167 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:34:27.686563 24164 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:34:27.686695 24165 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 18:34:27.686717 24158 server_base.cc:1034] running on GCE node
I20260806 18:34:27.687275 24158 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 18:34:27.687852 24158 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 18:34:27.689007 24158 hybrid_clock.cc:648] HybridClock initialized: now 1786041267688979 us; error 39 us; skew 500 ppm
Aug 06 18:34:27 dist-test-slave-1xrh krb5kdc[23905](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041267, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.194.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 18:34:27.691916 24158 init.cc:377] Logged in from keytab as kudu/127.19.194.2@KRBTEST.COM (short username kudu)
I20260806 18:34:27.692754 24158 webserver.cc:460] Webserver started at http://127.19.194.2:44069/ using document root <none> and password file <none>
I20260806 18:34:27.693310 24158 fs_manager.cc:359] Metadata directory not provided
I20260806 18:34:27.693372 24158 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 18:34:27.693588 24158 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 18:34:27.695573 24158 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786041246094603-20232-0/minicluster-data/ts-1/data/instance:
uuid: "723e6714cc964545bb518af4df3432e8"
format_stamp: "Formatted at 2026-08-06 18:34:27 on dist-test-slave-1xrh"
server_key: "a9e94d6b42f6696b7ea79dbd81a3df6a"
server_key_iv: "0774ef75bf750ea99a6564eb5048d364"
server_key_version: "encryptionkey@0"
I20260806 18:34:27.696147 24158 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786041246094603-20232-0/minicluster-data/ts-1/wal/instance:
uuid: "723e6714cc964545bb518af4df3432e8"
format_stamp: "Formatted at 2026-08-06 18:34:27 on dist-test-slave-1xrh"
server_key: "a9e94d6b42f6696b7ea79dbd81a3df6a"
server_key_iv: "0774ef75bf750ea99a6564eb5048d364"
server_key_version: "encryptionkey@0"
I20260806 18:34:27.699636 24158 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.001s
I20260806 18:34:27.701900 24174 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 18:34:27.703047 24158 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260806 18:34:27.703199 24158 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786041246094603-20232-0/minicluster-data/ts-1/data,/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786041246094603-20232-0/minicluster-data/ts-1/wal
uuid: "723e6714cc964545bb518af4df3432e8"
format_stamp: "Formatted at 2026-08-06 18:34:27 on dist-test-slave-1xrh"
server_key: "a9e94d6b42f6696b7ea79dbd81a3df6a"
server_key_iv: "0774ef75bf750ea99a6564eb5048d364"
server_key_version: "encryptionkey@0"
I20260806 18:34:27.703310 24158 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786041246094603-20232-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786041246094603-20232-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786041246094603-20232-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 18:34:27.715404 24158 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 18:34:27.718010 24158 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 18:34:27.718259 24158 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 18:34:27.718791 24158 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 18:34:27.719653 24158 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 18:34:27.719723 24158 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 18:34:27.719792 24158 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 18:34:27.719846 24158 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 18:34:27.728681 24158 rpc_server.cc:307] RPC server started. Bound to: 127.19.194.2:38755
I20260806 18:34:27.728709 24301 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.194.2:38755 every 8 connection(s)
I20260806 18:34:27.729507 24158 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786041246094603-20232-0/minicluster-data/ts-1/data/info.pb
I20260806 18:34:27.734834 20232 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJW69tn/build/debug/bin/kudu as pid 24158
I20260806 18:34:27.734980 20232 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786041246094603-20232-0/minicluster-data/ts-1/wal/instance
I20260806 18:34:27.735321 20232 external_mini_cluster.cc:1428] Setting key 83c3674168dc4341548db797ab89f540
Aug 06 18:34:27 dist-test-slave-1xrh krb5kdc[23905](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041267, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.194.2@KRBTEST.COM for kudu/127.19.194.62@KRBTEST.COM
I20260806 18:34:27.742949 23992 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:34:27.731285 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.194.2:55251 (local address 127.19.194.62:44971)
0806 18:34:27.731415 (+ 130us) server_negotiation.cc:206] Beginning negotiation
0806 18:34:27.731419 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 18:34:27.732213 (+ 794us) server_negotiation.cc:407] Connection header received
0806 18:34:27.733612 (+ 1399us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:34:27.733615 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:34:27.733674 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:34:27.733781 (+ 107us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:34:27.734926 (+ 1145us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:34:27.735613 (+ 687us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:34:27.736729 (+ 1116us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:34:27.736942 (+ 213us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:34:27.737056 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:34:27.739468 (+ 2412us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:34:27.739471 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:34:27.739473 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 18:34:27.739495 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:34:27.740941 (+ 1446us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:34:27.741453 (+ 512us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:34:27.741457 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:34:27.741459 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:34:27.741546 (+ 87us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:34:27.741913 (+ 367us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:34:27.741916 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:34:27.741918 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:34:27.742106 (+ 188us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:34:27.742211 (+ 105us) server_negotiation.cc:1035] Waiting for connection context
0806 18:34:27.742666 (+ 455us) server_negotiation.cc:299] Negotiation successful
0806 18:34:27.742783 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260806 18:34:27.743777 24304 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:34:27.731606 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.194.62:44971 (local address 127.19.194.2:55251)
0806 18:34:27.732071 (+ 465us) negotiation.cc:107] Waiting for socket to connect
0806 18:34:27.732111 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0806 18:34:27.733379 (+ 1268us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:34:27.733977 (+ 598us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:34:27.733983 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:34:27.734325 (+ 342us) client_negotiation.cc:189] Negotiated authn=SASL
0806 18:34:27.734780 (+ 455us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:34:27.734792 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:34:27.735739 (+ 947us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:34:27.735742 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:34:27.736596 (+ 854us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:34:27.736606 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:34:27.737090 (+ 484us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:34:27.737095 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:34:27.737249 (+ 154us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:34:27.737808 (+ 559us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 18:34:27.737824 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0806 18:34:27.739303 (+ 1479us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 18:34:27.741069 (+ 1766us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:34:27.741075 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:34:27.741089 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:34:27.741345 (+ 256us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:34:27.741664 (+ 319us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:34:27.741668 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:34:27.741670 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:34:27.741819 (+ 149us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:34:27.742213 (+ 394us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 18:34:27.742220 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 18:34:27.742439 (+ 219us) client_negotiation.cc:769] Sending connection context
0806 18:34:27.742619 (+ 180us) client_negotiation.cc:240] Negotiation successful
0806 18:34:27.742912 (+ 293us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":282,"thread_start_us":106,"threads_started":1}
WARNING: no policy specified for kudu/127.19.194.3@KRBTEST.COM; defaulting to no policy
I20260806 18:34:27.745113 24302 heartbeater.cc:347] Connected to a master server at 127.19.194.62:44971
I20260806 18:34:27.745374 24302 heartbeater.cc:464] Registering TS with master...
I20260806 18:34:27.745800 24302 heartbeater.cc:511] Master 127.19.194.62:44971 requested a full tablet report, sending...
I20260806 18:34:27.746816 23953 ts_manager.cc:194] Registered new tserver with Master: 723e6714cc964545bb518af4df3432e8 (127.19.194.2:38755)
I20260806 18:34:27.747453 23953 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.19.194.2@KRBTEST.COM'} at 127.19.194.2:55251
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.19.194.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.19.194.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786041246094603-20232-0/krb5kdc/kudu_127.19.194.3.keytab.
Entry for principal kudu/127.19.194.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786041246094603-20232-0/krb5kdc/kudu_127.19.194.3.keytab.
I20260806 18:34:27.762717 20232 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJW69tn/build/debug/bin/kudu
/tmp/dist-test-taskJW69tn/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786041246094603-20232-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786041246094603-20232-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786041246094603-20232-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786041246094603-20232-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.19.194.3:0
--local_ip_for_outbound_sockets=127.19.194.3
--webserver_interface=127.19.194.3
--webserver_port=0
--tserver_master_addrs=127.19.194.62:44971
--keytab_file=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786041246094603-20232-0/krb5kdc/kudu_127.19.194.3.keytab
--principal=kudu/127.19.194.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.19.194.20:34191
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786041246094603-20232-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786041246094603-20232-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786041246094603-20232-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 18:34:27.869803 24307 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 18:34:27.870035 24307 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 18:34:27.870075 24307 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 18:34:27.873917 24307 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 18:34:27.873973 24307 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 18:34:27.874063 24307 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.19.194.3
I20260806 18:34:27.879140 24307 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.194.20:34191
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786041246094603-20232-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786041246094603-20232-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786041246094603-20232-0/krb5kdc/kudu_127.19.194.3.keytab
--principal=kudu/127.19.194.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.19.194.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786041246094603-20232-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.19.194.3
--webserver_port=0
--tserver_master_addrs=127.19.194.62:44971
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24307
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.19.194.3
--log_dir=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786041246094603-20232-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 17:59:13 UTC on ccycloud.quasar-wvsnzx.root.comops.site
build id 4571028
I20260806 18:34:27.880052 24307 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 18:34:27.881002 24307 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 18:34:27.890201 24316 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:34:27.890228 24314 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:34:27.890228 24313 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 18:34:27.890280 24307 server_base.cc:1034] running on GCE node
I20260806 18:34:27.890884 24307 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 18:34:27.891636 24307 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 18:34:27.892800 24307 hybrid_clock.cc:648] HybridClock initialized: now 1786041267892757 us; error 55 us; skew 500 ppm
Aug 06 18:34:27 dist-test-slave-1xrh krb5kdc[23905](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041267, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.194.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 18:34:27.901268 24307 init.cc:377] Logged in from keytab as kudu/127.19.194.3@KRBTEST.COM (short username kudu)
I20260806 18:34:27.902065 24307 webserver.cc:460] Webserver started at http://127.19.194.3:35155/ using document root <none> and password file <none>
I20260806 18:34:27.902583 24307 fs_manager.cc:359] Metadata directory not provided
I20260806 18:34:27.902642 24307 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 18:34:27.902796 24307 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 18:34:27.904770 24307 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786041246094603-20232-0/minicluster-data/ts-2/data/instance:
uuid: "712868811dfd405ba22fa2812c623a35"
format_stamp: "Formatted at 2026-08-06 18:34:27 on dist-test-slave-1xrh"
server_key: "80c800568cf3386f1aafda54ae4a0b4b"
server_key_iv: "6dd0f16a6b9e27c060ac50b62e63acb3"
server_key_version: "encryptionkey@0"
I20260806 18:34:27.905257 24307 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786041246094603-20232-0/minicluster-data/ts-2/wal/instance:
uuid: "712868811dfd405ba22fa2812c623a35"
format_stamp: "Formatted at 2026-08-06 18:34:27 on dist-test-slave-1xrh"
server_key: "80c800568cf3386f1aafda54ae4a0b4b"
server_key_iv: "6dd0f16a6b9e27c060ac50b62e63acb3"
server_key_version: "encryptionkey@0"
I20260806 18:34:27.909062 24307 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.002s
I20260806 18:34:27.911397 24323 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 18:34:27.912492 24307 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260806 18:34:27.912618 24307 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786041246094603-20232-0/minicluster-data/ts-2/data,/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786041246094603-20232-0/minicluster-data/ts-2/wal
uuid: "712868811dfd405ba22fa2812c623a35"
format_stamp: "Formatted at 2026-08-06 18:34:27 on dist-test-slave-1xrh"
server_key: "80c800568cf3386f1aafda54ae4a0b4b"
server_key_iv: "6dd0f16a6b9e27c060ac50b62e63acb3"
server_key_version: "encryptionkey@0"
I20260806 18:34:27.912714 24307 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786041246094603-20232-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786041246094603-20232-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786041246094603-20232-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 18:34:27.933130 24307 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 18:34:27.936420 24307 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 18:34:27.936579 24307 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 18:34:27.937191 24307 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 18:34:27.937999 24307 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 18:34:27.938050 24307 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 18:34:27.938117 24307 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 18:34:27.938160 24307 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 18:34:27.946749 24307 rpc_server.cc:307] RPC server started. Bound to: 127.19.194.3:43899
I20260806 18:34:27.946791 24442 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.194.3:43899 every 8 connection(s)
I20260806 18:34:27.947635 24307 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786041246094603-20232-0/minicluster-data/ts-2/data/info.pb
I20260806 18:34:27.948921 20232 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJW69tn/build/debug/bin/kudu as pid 24307
I20260806 18:34:27.949060 20232 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786041246094603-20232-0/minicluster-data/ts-2/wal/instance
I20260806 18:34:27.949400 20232 external_mini_cluster.cc:1428] Setting key aae22a7ca6d912453085f07e84602161
Aug 06 18:34:27 dist-test-slave-1xrh krb5kdc[23905](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041267, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.194.3@KRBTEST.COM for kudu/127.19.194.62@KRBTEST.COM
I20260806 18:34:27.961225 23992 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:34:27.949488 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.194.3:49123 (local address 127.19.194.62:44971)
0806 18:34:27.949625 (+ 137us) server_negotiation.cc:206] Beginning negotiation
0806 18:34:27.949629 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 18:34:27.950605 (+ 976us) server_negotiation.cc:407] Connection header received
0806 18:34:27.952153 (+ 1548us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:34:27.952156 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:34:27.952201 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:34:27.952269 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:34:27.953490 (+ 1221us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:34:27.954083 (+ 593us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:34:27.955270 (+ 1187us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:34:27.955489 (+ 219us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:34:27.955569 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:34:27.957896 (+ 2327us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:34:27.957899 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:34:27.957901 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 18:34:27.957926 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:34:27.959443 (+ 1517us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:34:27.959918 (+ 475us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:34:27.959920 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:34:27.959922 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:34:27.960000 (+ 78us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:34:27.960294 (+ 294us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:34:27.960297 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:34:27.960298 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:34:27.960479 (+ 181us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:34:27.960553 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0806 18:34:27.961002 (+ 449us) server_negotiation.cc:299] Negotiation successful
0806 18:34:27.961101 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260806 18:34:27.962074 24445 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:34:27.949929 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.194.62:44971 (local address 127.19.194.3:49123)
0806 18:34:27.950451 (+ 522us) negotiation.cc:107] Waiting for socket to connect
0806 18:34:27.950501 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0806 18:34:27.951953 (+ 1452us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:34:27.952433 (+ 480us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:34:27.952440 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:34:27.952847 (+ 407us) client_negotiation.cc:189] Negotiated authn=SASL
0806 18:34:27.953321 (+ 474us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:34:27.953335 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:34:27.954194 (+ 859us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:34:27.954197 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:34:27.955111 (+ 914us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:34:27.955169 (+ 58us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:34:27.955619 (+ 450us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:34:27.955623 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:34:27.955759 (+ 136us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:34:27.956282 (+ 523us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 18:34:27.956298 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0806 18:34:27.957747 (+ 1449us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 18:34:27.959560 (+ 1813us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:34:27.959565 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:34:27.959576 (+ 11us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:34:27.959819 (+ 243us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:34:27.960081 (+ 262us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:34:27.960083 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:34:27.960084 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:34:27.960210 (+ 126us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:34:27.960574 (+ 364us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 18:34:27.960579 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 18:34:27.960798 (+ 219us) client_negotiation.cc:769] Sending connection context
0806 18:34:27.960952 (+ 154us) client_negotiation.cc:240] Negotiation successful
0806 18:34:27.961202 (+ 250us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":323,"thread_start_us":171,"threads_started":1}
I20260806 18:34:27.963340 24443 heartbeater.cc:347] Connected to a master server at 127.19.194.62:44971
I20260806 18:34:27.963605 24443 heartbeater.cc:464] Registering TS with master...
I20260806 18:34:27.964123 24443 heartbeater.cc:511] Master 127.19.194.62:44971 requested a full tablet report, sending...
I20260806 18:34:27.965155 23953 ts_manager.cc:194] Registered new tserver with Master: 712868811dfd405ba22fa2812c623a35 (127.19.194.3:43899)
I20260806 18:34:27.965727 23953 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.19.194.3@KRBTEST.COM'} at 127.19.194.3:49123
I20260806 18:34:27.975415 20232 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 06 18:34:27 dist-test-slave-1xrh krb5kdc[23905](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041267, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 06 18:34:27 dist-test-slave-1xrh krb5kdc[23905](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041267, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.19.194.1@KRBTEST.COM
I20260806 18:34:28.002319 24450 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:34:27.993022 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53232 (local address 127.19.194.1:42717)
0806 18:34:27.993325 (+ 303us) server_negotiation.cc:206] Beginning negotiation
0806 18:34:27.993340 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0806 18:34:27.993381 (+ 41us) server_negotiation.cc:407] Connection header received
0806 18:34:27.993431 (+ 50us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:34:27.993436 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:34:27.993627 (+ 191us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:34:27.993741 (+ 114us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:34:27.994359 (+ 618us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:34:27.995190 (+ 831us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:34:27.996003 (+ 813us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:34:27.996410 (+ 407us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:34:27.996494 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:34:27.998925 (+ 2431us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:34:27.998935 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:34:27.998947 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 18:34:27.998970 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:34:28.000482 (+ 1512us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:34:28.000917 (+ 435us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:34:28.000922 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:34:28.000923 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:34:28.001001 (+ 78us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:34:28.001322 (+ 321us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:34:28.001326 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:34:28.001328 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:34:28.001566 (+ 238us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:34:28.001782 (+ 216us) server_negotiation.cc:1035] Waiting for connection context
0806 18:34:28.001988 (+ 206us) server_negotiation.cc:299] Negotiation successful
0806 18:34:28.002151 (+ 163us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":194,"thread_start_us":94,"threads_started":1}
Aug 06 18:34:28 dist-test-slave-1xrh krb5kdc[23905](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041267, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.19.194.62@KRBTEST.COM
I20260806 18:34:28.013635 23992 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:34:28.005212 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44812 (local address 127.19.194.62:44971)
0806 18:34:28.005361 (+ 149us) server_negotiation.cc:206] Beginning negotiation
0806 18:34:28.005365 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 18:34:28.005379 (+ 14us) server_negotiation.cc:407] Connection header received
0806 18:34:28.005495 (+ 116us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:34:28.005497 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:34:28.005538 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:34:28.005604 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:34:28.006200 (+ 596us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:34:28.006705 (+ 505us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:34:28.007503 (+ 798us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:34:28.007705 (+ 202us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:34:28.007783 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:34:28.010309 (+ 2526us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:34:28.010314 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:34:28.010319 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 18:34:28.010351 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:34:28.011930 (+ 1579us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:34:28.012353 (+ 423us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:34:28.012357 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:34:28.012358 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:34:28.012442 (+ 84us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:34:28.012718 (+ 276us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:34:28.012722 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:34:28.012724 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:34:28.012947 (+ 223us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:34:28.013090 (+ 143us) server_negotiation.cc:1035] Waiting for connection context
0806 18:34:28.013320 (+ 230us) server_negotiation.cc:299] Negotiation successful
0806 18:34:28.013448 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
W20260806 18:34:28.014011 23953 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.TSHeartbeat from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:44812
I20260806 18:34:28.014627 20232 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJW69tn/build/debug/bin/kudu with pid 24018
I20260806 18:34:28.024809 20232 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJW69tn/build/debug/bin/kudu with pid 24158
I20260806 18:34:28.031375 20232 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJW69tn/build/debug/bin/kudu with pid 24307
I20260806 18:34:28.037494 20232 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJW69tn/build/debug/bin/kudu with pid 23918
2026-08-06T18:34:28Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizedSuperuser (2900 ms)
[ RUN ] SecurityITest.TestDisableWebUI
Loading random data
Initializing database '/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786041246094603-20232-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 18:34:28 dist-test-slave-1xrh krb5kdc[24455](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 18:34:28 dist-test-slave-1xrh krb5kdc[24455](info): set up 2 sockets
Aug 06 18:34:28 dist-test-slave-1xrh krb5kdc[24455](info): commencing operation
krb5kdc: starting...
W20260806 18:34:29.113034 20232 mini_kdc.cc:121] Time spent starting KDC: real 1.053s user 0.003s sys 0.003s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786041246094603-20232-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786041246094603-20232-0/krb5kdc/test-user.keytab.
Aug 06 18:34:29 dist-test-slave-1xrh krb5kdc[24455](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041269, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T18:34:29Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T18:34:29Z Disabled control of system clock
WARNING: no policy specified for kudu/127.19.194.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.19.194.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.19.194.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786041246094603-20232-0/krb5kdc/kudu_127.19.194.62.keytab.
Entry for principal kudu/127.19.194.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786041246094603-20232-0/krb5kdc/kudu_127.19.194.62.keytab.
I20260806 18:34:29.234318 20232 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJW69tn/build/debug/bin/kudu
/tmp/dist-test-taskJW69tn/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786041246094603-20232-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786041246094603-20232-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786041246094603-20232-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786041246094603-20232-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.19.194.62:38703
--webserver_interface=127.19.194.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786041246094603-20232-0/krb5kdc/kudu_127.19.194.62.keytab
--principal=kudu/127.19.194.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.19.194.20:33005
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.19.194.62:38703
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786041246094603-20232-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786041246094603-20232-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786041246094603-20232-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 18:34:29.346597 24469 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 18:34:29.346824 24469 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 18:34:29.346906 24469 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 18:34:29.350924 24469 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 18:34:29.350975 24469 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 18:34:29.351016 24469 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 18:34:29.351054 24469 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 18:34:29.351079 24469 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 18:34:29.356081 24469 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.194.20:33005
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786041246094603-20232-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786041246094603-20232-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.19.194.62:38703
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786041246094603-20232-0/krb5kdc/kudu_127.19.194.62.keytab
--principal=kudu/127.19.194.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.19.194.62:38703
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786041246094603-20232-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.19.194.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24469
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786041246094603-20232-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 17:59:13 UTC on ccycloud.quasar-wvsnzx.root.comops.site
build id 4571028
I20260806 18:34:29.357023 24469 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 18:34:29.357986 24469 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 18:34:29.366142 24474 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:34:29.366147 24477 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 18:34:29.366242 24469 server_base.cc:1034] running on GCE node
W20260806 18:34:29.366135 24475 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 18:34:29.366990 24469 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 18:34:29.368090 24469 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 18:34:29.369283 24469 hybrid_clock.cc:648] HybridClock initialized: now 1786041269369265 us; error 29 us; skew 500 ppm
Aug 06 18:34:29 dist-test-slave-1xrh krb5kdc[24455](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041269, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.194.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 18:34:29.372010 24469 init.cc:377] Logged in from keytab as kudu/127.19.194.62@KRBTEST.COM (short username kudu)
I20260806 18:34:29.372702 24469 fs_manager.cc:359] Metadata directory not provided
I20260806 18:34:29.372798 24469 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 18:34:29.373064 24469 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 18:34:29.375065 24469 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786041246094603-20232-0/minicluster-data/master-0/data/instance:
uuid: "eca707e6ea054f71a1aaaa6463831932"
format_stamp: "Formatted at 2026-08-06 18:34:29 on dist-test-slave-1xrh"
server_key: "97a55726a69cb7a24935cb0f180f1e65"
server_key_iv: "104ff832826a7270012c93fb229b8812"
server_key_version: "encryptionkey@0"
I20260806 18:34:29.375689 24469 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786041246094603-20232-0/minicluster-data/master-0/wal/instance:
uuid: "eca707e6ea054f71a1aaaa6463831932"
format_stamp: "Formatted at 2026-08-06 18:34:29 on dist-test-slave-1xrh"
server_key: "97a55726a69cb7a24935cb0f180f1e65"
server_key_iv: "104ff832826a7270012c93fb229b8812"
server_key_version: "encryptionkey@0"
I20260806 18:34:29.379506 24469 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260806 18:34:29.381808 24483 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 18:34:29.382958 24469 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260806 18:34:29.383165 24469 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786041246094603-20232-0/minicluster-data/master-0/data,/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786041246094603-20232-0/minicluster-data/master-0/wal
uuid: "eca707e6ea054f71a1aaaa6463831932"
format_stamp: "Formatted at 2026-08-06 18:34:29 on dist-test-slave-1xrh"
server_key: "97a55726a69cb7a24935cb0f180f1e65"
server_key_iv: "104ff832826a7270012c93fb229b8812"
server_key_version: "encryptionkey@0"
I20260806 18:34:29.383283 24469 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786041246094603-20232-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786041246094603-20232-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786041246094603-20232-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 18:34:29.392457 24469 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 18:34:29.394972 24469 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 18:34:29.395249 24469 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 18:34:29.402150 24469 rpc_server.cc:307] RPC server started. Bound to: 127.19.194.62:38703
I20260806 18:34:29.402405 24538 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.194.62:38703 every 8 connection(s)
I20260806 18:34:29.402789 24469 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786041246094603-20232-0/minicluster-data/master-0/data/info.pb
I20260806 18:34:29.405390 24539 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 18:34:29.409812 20232 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJW69tn/build/debug/bin/kudu as pid 24469
I20260806 18:34:29.409948 20232 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786041246094603-20232-0/minicluster-data/master-0/wal/instance
I20260806 18:34:29.410264 20232 external_mini_cluster.cc:1428] Setting key bd8f7d0c8cb69d88631fe1253225344f
I20260806 18:34:29.411089 24539 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P eca707e6ea054f71a1aaaa6463831932: Bootstrap starting.
I20260806 18:34:29.413385 24539 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P eca707e6ea054f71a1aaaa6463831932: Neither blocks nor log segments found. Creating new log.
I20260806 18:34:29.414287 24539 log.cc:824] T 00000000000000000000000000000000 P eca707e6ea054f71a1aaaa6463831932: Log is configured to *not* fsync() on all Append() calls
I20260806 18:34:29.416465 24539 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P eca707e6ea054f71a1aaaa6463831932: No bootstrap required, opened a new log
Aug 06 18:34:29 dist-test-slave-1xrh krb5kdc[24455](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041269, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.19.194.62@KRBTEST.COM
I20260806 18:34:29.420991 24539 raft_consensus.cc:348] T 00000000000000000000000000000000 P eca707e6ea054f71a1aaaa6463831932 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "eca707e6ea054f71a1aaaa6463831932" member_type: VOTER last_known_addr { host: "127.19.194.62" port: 38703 } }
I20260806 18:34:29.421248 24539 raft_consensus.cc:374] T 00000000000000000000000000000000 P eca707e6ea054f71a1aaaa6463831932 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 18:34:29.421360 24539 raft_consensus.cc:729] T 00000000000000000000000000000000 P eca707e6ea054f71a1aaaa6463831932 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: eca707e6ea054f71a1aaaa6463831932, State: Initialized, Role: FOLLOWER
I20260806 18:34:29.421768 24539 consensus_queue.cc:260] T 00000000000000000000000000000000 P eca707e6ea054f71a1aaaa6463831932 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "eca707e6ea054f71a1aaaa6463831932" member_type: VOTER last_known_addr { host: "127.19.194.62" port: 38703 } }
I20260806 18:34:29.421887 24539 raft_consensus.cc:388] T 00000000000000000000000000000000 P eca707e6ea054f71a1aaaa6463831932 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 18:34:29.421972 24539 raft_consensus.cc:482] T 00000000000000000000000000000000 P eca707e6ea054f71a1aaaa6463831932 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 18:34:29.422086 24539 raft_consensus.cc:3037] T 00000000000000000000000000000000 P eca707e6ea054f71a1aaaa6463831932 [term 0 FOLLOWER]: Advancing to term 1
I20260806 18:34:29.423313 24539 raft_consensus.cc:504] T 00000000000000000000000000000000 P eca707e6ea054f71a1aaaa6463831932 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "eca707e6ea054f71a1aaaa6463831932" member_type: VOTER last_known_addr { host: "127.19.194.62" port: 38703 } }
I20260806 18:34:29.423673 24539 leader_election.cc:302] T 00000000000000000000000000000000 P eca707e6ea054f71a1aaaa6463831932 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: eca707e6ea054f71a1aaaa6463831932; no voters:
I20260806 18:34:29.424058 24539 leader_election.cc:288] T 00000000000000000000000000000000 P eca707e6ea054f71a1aaaa6463831932 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 18:34:29.424156 24544 raft_consensus.cc:2781] T 00000000000000000000000000000000 P eca707e6ea054f71a1aaaa6463831932 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 18:34:29.424441 24544 raft_consensus.cc:686] T 00000000000000000000000000000000 P eca707e6ea054f71a1aaaa6463831932 [term 1 LEADER]: Becoming Leader. State: Replica: eca707e6ea054f71a1aaaa6463831932, State: Running, Role: LEADER
I20260806 18:34:29.424804 24544 consensus_queue.cc:237] T 00000000000000000000000000000000 P eca707e6ea054f71a1aaaa6463831932 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "eca707e6ea054f71a1aaaa6463831932" member_type: VOTER last_known_addr { host: "127.19.194.62" port: 38703 } }
I20260806 18:34:29.425340 24542 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:34:29.411612 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42582 (local address 127.19.194.62:38703)
0806 18:34:29.412006 (+ 394us) server_negotiation.cc:206] Beginning negotiation
0806 18:34:29.412021 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0806 18:34:29.412054 (+ 33us) server_negotiation.cc:407] Connection header received
0806 18:34:29.412625 (+ 571us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:34:29.412641 (+ 16us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:34:29.412930 (+ 289us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:34:29.413207 (+ 277us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:34:29.414172 (+ 965us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:34:29.415192 (+ 1020us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:34:29.416322 (+ 1130us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:34:29.416814 (+ 492us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:34:29.416928 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:34:29.419990 (+ 3062us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:34:29.419995 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:34:29.420008 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 18:34:29.420034 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:34:29.422070 (+ 2036us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:34:29.422575 (+ 505us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:34:29.422580 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:34:29.422585 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:34:29.422693 (+ 108us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:34:29.423011 (+ 318us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:34:29.423013 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:34:29.423014 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:34:29.423510 (+ 496us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:34:29.423660 (+ 150us) server_negotiation.cc:1035] Waiting for connection context
0806 18:34:29.424075 (+ 415us) server_negotiation.cc:299] Negotiation successful
0806 18:34:29.424392 (+ 317us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":253,"thread_start_us":124,"threads_started":1}
I20260806 18:34:29.425443 24539 sys_catalog.cc:564] T 00000000000000000000000000000000 P eca707e6ea054f71a1aaaa6463831932 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 18:34:29.427254 24545 sys_catalog.cc:455] T 00000000000000000000000000000000 P eca707e6ea054f71a1aaaa6463831932 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "eca707e6ea054f71a1aaaa6463831932" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "eca707e6ea054f71a1aaaa6463831932" member_type: VOTER last_known_addr { host: "127.19.194.62" port: 38703 } } }
I20260806 18:34:29.427373 24545 sys_catalog.cc:458] T 00000000000000000000000000000000 P eca707e6ea054f71a1aaaa6463831932 [sys.catalog]: This master's current role is: LEADER
I20260806 18:34:29.427709 24546 sys_catalog.cc:455] T 00000000000000000000000000000000 P eca707e6ea054f71a1aaaa6463831932 [sys.catalog]: SysCatalogTable state changed. Reason: New leader eca707e6ea054f71a1aaaa6463831932. Latest consensus state: current_term: 1 leader_uuid: "eca707e6ea054f71a1aaaa6463831932" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "eca707e6ea054f71a1aaaa6463831932" member_type: VOTER last_known_addr { host: "127.19.194.62" port: 38703 } } }
I20260806 18:34:29.428046 24546 sys_catalog.cc:458] T 00000000000000000000000000000000 P eca707e6ea054f71a1aaaa6463831932 [sys.catalog]: This master's current role is: LEADER
W20260806 18:34:29.430053 24560 catalog_manager.cc:1558] T 00000000000000000000000000000000 P eca707e6ea054f71a1aaaa6463831932: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260806 18:34:29.430120 24560 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260806 18:34:29.430202 24559 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 18:34:29.430888 24559 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 18:34:29.435869 24559 catalog_manager.cc:1347] Generated new cluster ID: f7aeaacacc864922afd16ae43f68a50c
I20260806 18:34:29.435930 24559 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 18:34:29.452023 24559 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 18:34:29.453166 24559 catalog_manager.cc:1504] Loading token signing keys...
I20260806 18:34:29.461835 24559 catalog_manager.cc:5868] T 00000000000000000000000000000000 P eca707e6ea054f71a1aaaa6463831932: Generated new TSK 0
I20260806 18:34:29.462520 24559 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.19.194.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.19.194.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.19.194.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786041246094603-20232-0/krb5kdc/kudu_127.19.194.1.keytab.
Entry for principal kudu/127.19.194.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786041246094603-20232-0/krb5kdc/kudu_127.19.194.1.keytab.
I20260806 18:34:29.494166 20232 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJW69tn/build/debug/bin/kudu
/tmp/dist-test-taskJW69tn/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786041246094603-20232-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786041246094603-20232-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786041246094603-20232-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786041246094603-20232-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.19.194.1:0
--local_ip_for_outbound_sockets=127.19.194.1
--webserver_interface=127.19.194.1
--webserver_port=0
--tserver_master_addrs=127.19.194.62:38703
--keytab_file=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786041246094603-20232-0/krb5kdc/kudu_127.19.194.1.keytab
--principal=kudu/127.19.194.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.19.194.20:33005
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786041246094603-20232-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786041246094603-20232-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786041246094603-20232-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 18:34:29.602454 24566 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 18:34:29.602720 24566 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 18:34:29.602802 24566 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 18:34:29.606923 24566 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 18:34:29.607010 24566 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 18:34:29.607115 24566 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.19.194.1
I20260806 18:34:29.612293 24566 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.194.20:33005
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786041246094603-20232-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786041246094603-20232-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786041246094603-20232-0/krb5kdc/kudu_127.19.194.1.keytab
--principal=kudu/127.19.194.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.19.194.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786041246094603-20232-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.19.194.1
--webserver_port=0
--tserver_master_addrs=127.19.194.62:38703
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24566
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.19.194.1
--log_dir=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786041246094603-20232-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 17:59:13 UTC on ccycloud.quasar-wvsnzx.root.comops.site
build id 4571028
I20260806 18:34:29.613289 24566 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 18:34:29.614293 24566 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260806 18:34:29.623435 24566 server_base.cc:1034] running on GCE node
W20260806 18:34:29.623582 24574 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:34:29.623587 24571 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:34:29.623611 24572 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 18:34:29.624105 24566 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 18:34:29.624717 24566 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 18:34:29.625872 24566 hybrid_clock.cc:648] HybridClock initialized: now 1786041269625852 us; error 32 us; skew 500 ppm
Aug 06 18:34:29 dist-test-slave-1xrh krb5kdc[24455](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041269, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.194.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 18:34:29.628793 24566 init.cc:377] Logged in from keytab as kudu/127.19.194.1@KRBTEST.COM (short username kudu)
I20260806 18:34:29.629518 24566 fs_manager.cc:359] Metadata directory not provided
I20260806 18:34:29.629611 24566 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 18:34:29.629853 24566 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 18:34:29.631891 24566 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786041246094603-20232-0/minicluster-data/ts-0/data/instance:
uuid: "03b21c075a2c446dbf938ad590297391"
format_stamp: "Formatted at 2026-08-06 18:34:29 on dist-test-slave-1xrh"
server_key: "735c4ad2a59cbe0a083a3faec513f063"
server_key_iv: "8bb6aaf660eb4e57496d258452af9f0a"
server_key_version: "encryptionkey@0"
I20260806 18:34:29.632436 24566 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786041246094603-20232-0/minicluster-data/ts-0/wal/instance:
uuid: "03b21c075a2c446dbf938ad590297391"
format_stamp: "Formatted at 2026-08-06 18:34:29 on dist-test-slave-1xrh"
server_key: "735c4ad2a59cbe0a083a3faec513f063"
server_key_iv: "8bb6aaf660eb4e57496d258452af9f0a"
server_key_version: "encryptionkey@0"
I20260806 18:34:29.636019 24566 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.001s
I20260806 18:34:29.638248 24580 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 18:34:29.639451 24566 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260806 18:34:29.639586 24566 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786041246094603-20232-0/minicluster-data/ts-0/data,/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786041246094603-20232-0/minicluster-data/ts-0/wal
uuid: "03b21c075a2c446dbf938ad590297391"
format_stamp: "Formatted at 2026-08-06 18:34:29 on dist-test-slave-1xrh"
server_key: "735c4ad2a59cbe0a083a3faec513f063"
server_key_iv: "8bb6aaf660eb4e57496d258452af9f0a"
server_key_version: "encryptionkey@0"
I20260806 18:34:29.639691 24566 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786041246094603-20232-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786041246094603-20232-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786041246094603-20232-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 18:34:29.654053 24566 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 18:34:29.657240 24566 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 18:34:29.657440 24566 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 18:34:29.658004 24566 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 18:34:29.658785 24566 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 18:34:29.658855 24566 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 18:34:29.658922 24566 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 18:34:29.658975 24566 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 18:34:29.667645 24566 rpc_server.cc:307] RPC server started. Bound to: 127.19.194.1:45271
I20260806 18:34:29.667666 24701 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.194.1:45271 every 8 connection(s)
I20260806 18:34:29.668175 24566 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786041246094603-20232-0/minicluster-data/ts-0/data/info.pb
I20260806 18:34:29.670414 20232 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJW69tn/build/debug/bin/kudu as pid 24566
I20260806 18:34:29.670554 20232 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786041246094603-20232-0/minicluster-data/ts-0/wal/instance
I20260806 18:34:29.670845 20232 external_mini_cluster.cc:1428] Setting key 597660f88fb6942022101584ef39da49
Aug 06 18:34:29 dist-test-slave-1xrh krb5kdc[24455](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041269, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.194.1@KRBTEST.COM for kudu/127.19.194.62@KRBTEST.COM
WARNING: no policy specified for kudu/127.19.194.2@KRBTEST.COM; defaulting to no policy
I20260806 18:34:29.683455 24542 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:34:29.669880 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.194.1:41607 (local address 127.19.194.62:38703)
0806 18:34:29.670013 (+ 133us) server_negotiation.cc:206] Beginning negotiation
0806 18:34:29.670017 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 18:34:29.670972 (+ 955us) server_negotiation.cc:407] Connection header received
0806 18:34:29.672500 (+ 1528us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:34:29.672504 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:34:29.672558 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:34:29.672649 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:34:29.673906 (+ 1257us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:34:29.674637 (+ 731us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:34:29.676434 (+ 1797us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:34:29.676662 (+ 228us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:34:29.676745 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:34:29.679718 (+ 2973us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:34:29.679722 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:34:29.679729 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 18:34:29.679754 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:34:29.681219 (+ 1465us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:34:29.681769 (+ 550us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:34:29.681774 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:34:29.681775 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:34:29.681850 (+ 75us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:34:29.682239 (+ 389us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:34:29.682242 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:34:29.682243 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:34:29.682416 (+ 173us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:34:29.682528 (+ 112us) server_negotiation.cc:1035] Waiting for connection context
0806 18:34:29.683059 (+ 531us) server_negotiation.cc:299] Negotiation successful
0806 18:34:29.683246 (+ 187us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.19.194.2@KRBTEST.COM" created.
I20260806 18:34:29.684288 24704 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:34:29.670262 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.194.62:38703 (local address 127.19.194.1:41607)
0806 18:34:29.670749 (+ 487us) negotiation.cc:107] Waiting for socket to connect
0806 18:34:29.670799 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0806 18:34:29.672246 (+ 1447us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:34:29.672835 (+ 589us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:34:29.672844 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:34:29.673279 (+ 435us) client_negotiation.cc:189] Negotiated authn=SASL
0806 18:34:29.673743 (+ 464us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:34:29.673756 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:34:29.674775 (+ 1019us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:34:29.674782 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:34:29.676301 (+ 1519us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:34:29.676316 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:34:29.676831 (+ 515us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:34:29.676838 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:34:29.677085 (+ 247us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:34:29.677705 (+ 620us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 18:34:29.677731 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0806 18:34:29.679496 (+ 1765us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 18:34:29.681344 (+ 1848us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:34:29.681350 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:34:29.681365 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:34:29.681648 (+ 283us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:34:29.681945 (+ 297us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:34:29.681949 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:34:29.681951 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:34:29.682092 (+ 141us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:34:29.682541 (+ 449us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 18:34:29.682549 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 18:34:29.682791 (+ 242us) client_negotiation.cc:769] Sending connection context
0806 18:34:29.682991 (+ 200us) client_negotiation.cc:240] Negotiation successful
0806 18:34:29.683330 (+ 339us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":275,"thread_start_us":136,"threads_started":1}
I20260806 18:34:29.685616 24702 heartbeater.cc:347] Connected to a master server at 127.19.194.62:38703
I20260806 18:34:29.685846 24702 heartbeater.cc:464] Registering TS with master...
I20260806 18:34:29.686359 24702 heartbeater.cc:511] Master 127.19.194.62:38703 requested a full tablet report, sending...
I20260806 18:34:29.688129 24500 ts_manager.cc:194] Registered new tserver with Master: 03b21c075a2c446dbf938ad590297391 (127.19.194.1:45271)
I20260806 18:34:29.689884 24500 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.19.194.1@KRBTEST.COM'} at 127.19.194.1:41607
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.19.194.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786041246094603-20232-0/krb5kdc/kudu_127.19.194.2.keytab.
Entry for principal kudu/127.19.194.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786041246094603-20232-0/krb5kdc/kudu_127.19.194.2.keytab.
I20260806 18:34:29.701771 20232 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJW69tn/build/debug/bin/kudu
/tmp/dist-test-taskJW69tn/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786041246094603-20232-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786041246094603-20232-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786041246094603-20232-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786041246094603-20232-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.19.194.2:0
--local_ip_for_outbound_sockets=127.19.194.2
--webserver_interface=127.19.194.2
--webserver_port=0
--tserver_master_addrs=127.19.194.62:38703
--keytab_file=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786041246094603-20232-0/krb5kdc/kudu_127.19.194.2.keytab
--principal=kudu/127.19.194.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.19.194.20:33005
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786041246094603-20232-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786041246094603-20232-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786041246094603-20232-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 18:34:29.810127 24708 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 18:34:29.810364 24708 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 18:34:29.810405 24708 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 18:34:29.814327 24708 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 18:34:29.814385 24708 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 18:34:29.814441 24708 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.19.194.2
I20260806 18:34:29.819556 24708 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.194.20:33005
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786041246094603-20232-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786041246094603-20232-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786041246094603-20232-0/krb5kdc/kudu_127.19.194.2.keytab
--principal=kudu/127.19.194.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.19.194.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786041246094603-20232-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.19.194.2
--webserver_port=0
--tserver_master_addrs=127.19.194.62:38703
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24708
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.19.194.2
--log_dir=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786041246094603-20232-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 17:59:13 UTC on ccycloud.quasar-wvsnzx.root.comops.site
build id 4571028
I20260806 18:34:29.820549 24708 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 18:34:29.821533 24708 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 18:34:29.830845 24713 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:34:29.830875 24714 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:34:29.830845 24716 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 18:34:29.831079 24708 server_base.cc:1034] running on GCE node
I20260806 18:34:29.831530 24708 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 18:34:29.832166 24708 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 18:34:29.833403 24708 hybrid_clock.cc:648] HybridClock initialized: now 1786041269833383 us; error 30 us; skew 500 ppm
Aug 06 18:34:29 dist-test-slave-1xrh krb5kdc[24455](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041269, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.194.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 18:34:29.836265 24708 init.cc:377] Logged in from keytab as kudu/127.19.194.2@KRBTEST.COM (short username kudu)
I20260806 18:34:29.836943 24708 fs_manager.cc:359] Metadata directory not provided
I20260806 18:34:29.837035 24708 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 18:34:29.837260 24708 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 18:34:29.839342 24708 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786041246094603-20232-0/minicluster-data/ts-1/data/instance:
uuid: "16113fc2ba3941e180858ae6fd9c79af"
format_stamp: "Formatted at 2026-08-06 18:34:29 on dist-test-slave-1xrh"
server_key: "d6b2608657af02f60e6f5e2a5d062ed9"
server_key_iv: "45dff870c02c0fa73ef3d290e78026f0"
server_key_version: "encryptionkey@0"
I20260806 18:34:29.839915 24708 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786041246094603-20232-0/minicluster-data/ts-1/wal/instance:
uuid: "16113fc2ba3941e180858ae6fd9c79af"
format_stamp: "Formatted at 2026-08-06 18:34:29 on dist-test-slave-1xrh"
server_key: "d6b2608657af02f60e6f5e2a5d062ed9"
server_key_iv: "45dff870c02c0fa73ef3d290e78026f0"
server_key_version: "encryptionkey@0"
I20260806 18:34:29.843506 24708 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20260806 18:34:29.845784 24723 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 18:34:29.846916 24708 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260806 18:34:29.847052 24708 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786041246094603-20232-0/minicluster-data/ts-1/data,/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786041246094603-20232-0/minicluster-data/ts-1/wal
uuid: "16113fc2ba3941e180858ae6fd9c79af"
format_stamp: "Formatted at 2026-08-06 18:34:29 on dist-test-slave-1xrh"
server_key: "d6b2608657af02f60e6f5e2a5d062ed9"
server_key_iv: "45dff870c02c0fa73ef3d290e78026f0"
server_key_version: "encryptionkey@0"
I20260806 18:34:29.847184 24708 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786041246094603-20232-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786041246094603-20232-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786041246094603-20232-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 18:34:29.862435 24708 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 18:34:29.865633 24708 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 18:34:29.865831 24708 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 18:34:29.866469 24708 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 18:34:29.867322 24708 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 18:34:29.867391 24708 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 18:34:29.867483 24708 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 18:34:29.867519 24708 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 18:34:29.876508 24708 rpc_server.cc:307] RPC server started. Bound to: 127.19.194.2:34921
I20260806 18:34:29.876529 24847 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.194.2:34921 every 8 connection(s)
I20260806 18:34:29.877110 24708 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786041246094603-20232-0/minicluster-data/ts-1/data/info.pb
I20260806 18:34:29.877729 20232 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJW69tn/build/debug/bin/kudu as pid 24708
I20260806 18:34:29.877862 20232 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786041246094603-20232-0/minicluster-data/ts-1/wal/instance
I20260806 18:34:29.878212 20232 external_mini_cluster.cc:1428] Setting key fc984aac7d8528dc24457400772c04f3
Aug 06 18:34:29 dist-test-slave-1xrh krb5kdc[24455](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041269, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.194.2@KRBTEST.COM for kudu/127.19.194.62@KRBTEST.COM
WARNING: no policy specified for kudu/127.19.194.3@KRBTEST.COM; defaulting to no policy
I20260806 18:34:29.891587 24542 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:34:29.878999 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.194.2:39407 (local address 127.19.194.62:38703)
0806 18:34:29.879170 (+ 171us) server_negotiation.cc:206] Beginning negotiation
0806 18:34:29.879175 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 18:34:29.880148 (+ 973us) server_negotiation.cc:407] Connection header received
0806 18:34:29.881556 (+ 1408us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:34:29.881560 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:34:29.881613 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:34:29.881692 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:34:29.883260 (+ 1568us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:34:29.883796 (+ 536us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:34:29.884965 (+ 1169us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:34:29.885192 (+ 227us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:34:29.885298 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:34:29.887980 (+ 2682us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:34:29.887984 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:34:29.887994 (+ 10us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 18:34:29.888019 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:34:29.889440 (+ 1421us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:34:29.889957 (+ 517us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:34:29.889961 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:34:29.889962 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:34:29.890050 (+ 88us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:34:29.890447 (+ 397us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:34:29.890450 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:34:29.890452 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:34:29.890644 (+ 192us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:34:29.890755 (+ 111us) server_negotiation.cc:1035] Waiting for connection context
0806 18:34:29.891285 (+ 530us) server_negotiation.cc:299] Negotiation successful
0806 18:34:29.891406 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.19.194.3@KRBTEST.COM" created.
I20260806 18:34:29.892485 24850 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:34:29.879499 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.194.62:38703 (local address 127.19.194.2:39407)
0806 18:34:29.880013 (+ 514us) negotiation.cc:107] Waiting for socket to connect
0806 18:34:29.880054 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0806 18:34:29.881349 (+ 1295us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:34:29.881863 (+ 514us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:34:29.881873 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:34:29.882394 (+ 521us) client_negotiation.cc:189] Negotiated authn=SASL
0806 18:34:29.883082 (+ 688us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:34:29.883104 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:34:29.883926 (+ 822us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:34:29.883928 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:34:29.884825 (+ 897us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:34:29.884835 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:34:29.885348 (+ 513us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:34:29.885352 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:34:29.885515 (+ 163us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:34:29.886142 (+ 627us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 18:34:29.886170 (+ 28us) client_negotiation.cc:656] calling sasl_client_start()
0806 18:34:29.887817 (+ 1647us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 18:34:29.889569 (+ 1752us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:34:29.889575 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:34:29.889589 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:34:29.889841 (+ 252us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:34:29.890144 (+ 303us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:34:29.890148 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:34:29.890150 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:34:29.890298 (+ 148us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:34:29.890753 (+ 455us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 18:34:29.890760 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 18:34:29.891002 (+ 242us) client_negotiation.cc:769] Sending connection context
0806 18:34:29.891222 (+ 220us) client_negotiation.cc:240] Negotiation successful
0806 18:34:29.891614 (+ 392us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":285,"thread_start_us":133,"threads_started":1}
I20260806 18:34:29.893755 24848 heartbeater.cc:347] Connected to a master server at 127.19.194.62:38703
I20260806 18:34:29.894022 24848 heartbeater.cc:464] Registering TS with master...
I20260806 18:34:29.894572 24848 heartbeater.cc:511] Master 127.19.194.62:38703 requested a full tablet report, sending...
I20260806 18:34:29.895766 24500 ts_manager.cc:194] Registered new tserver with Master: 16113fc2ba3941e180858ae6fd9c79af (127.19.194.2:34921)
I20260806 18:34:29.896348 24500 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.19.194.2@KRBTEST.COM'} at 127.19.194.2:39407
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.19.194.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786041246094603-20232-0/krb5kdc/kudu_127.19.194.3.keytab.
Entry for principal kudu/127.19.194.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786041246094603-20232-0/krb5kdc/kudu_127.19.194.3.keytab.
I20260806 18:34:29.909807 20232 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJW69tn/build/debug/bin/kudu
/tmp/dist-test-taskJW69tn/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786041246094603-20232-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786041246094603-20232-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786041246094603-20232-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786041246094603-20232-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.19.194.3:0
--local_ip_for_outbound_sockets=127.19.194.3
--webserver_interface=127.19.194.3
--webserver_port=0
--tserver_master_addrs=127.19.194.62:38703
--keytab_file=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786041246094603-20232-0/krb5kdc/kudu_127.19.194.3.keytab
--principal=kudu/127.19.194.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.19.194.20:33005
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786041246094603-20232-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786041246094603-20232-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786041246094603-20232-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 18:34:30.019142 24853 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 18:34:30.019398 24853 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 18:34:30.019477 24853 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 18:34:30.023583 24853 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 18:34:30.023644 24853 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 18:34:30.023733 24853 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.19.194.3
I20260806 18:34:30.029481 24853 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.194.20:33005
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786041246094603-20232-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786041246094603-20232-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786041246094603-20232-0/krb5kdc/kudu_127.19.194.3.keytab
--principal=kudu/127.19.194.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.19.194.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786041246094603-20232-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.19.194.3
--webserver_port=0
--tserver_master_addrs=127.19.194.62:38703
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24853
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.19.194.3
--log_dir=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786041246094603-20232-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 17:59:13 UTC on ccycloud.quasar-wvsnzx.root.comops.site
build id 4571028
I20260806 18:34:30.030545 24853 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 18:34:30.031630 24853 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 18:34:30.041652 24859 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:34:30.041669 24860 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:34:30.041672 24862 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 18:34:30.041961 24853 server_base.cc:1034] running on GCE node
I20260806 18:34:30.042311 24853 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 18:34:30.042959 24853 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 18:34:30.044137 24853 hybrid_clock.cc:648] HybridClock initialized: now 1786041270044125 us; error 26 us; skew 500 ppm
Aug 06 18:34:30 dist-test-slave-1xrh krb5kdc[24455](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041270, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.194.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 18:34:30.046816 24853 init.cc:377] Logged in from keytab as kudu/127.19.194.3@KRBTEST.COM (short username kudu)
I20260806 18:34:30.047580 24853 fs_manager.cc:359] Metadata directory not provided
I20260806 18:34:30.047686 24853 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 18:34:30.047907 24853 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 18:34:30.049911 24853 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786041246094603-20232-0/minicluster-data/ts-2/data/instance:
uuid: "3c05aac2e9314c70a8229a9f3f55ba82"
format_stamp: "Formatted at 2026-08-06 18:34:30 on dist-test-slave-1xrh"
server_key: "ee9e5dc9b174558e2cd47bbd92c416e2"
server_key_iv: "e3d39cb8c42c1198b2291af70110e7a6"
server_key_version: "encryptionkey@0"
I20260806 18:34:30.050498 24853 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786041246094603-20232-0/minicluster-data/ts-2/wal/instance:
uuid: "3c05aac2e9314c70a8229a9f3f55ba82"
format_stamp: "Formatted at 2026-08-06 18:34:30 on dist-test-slave-1xrh"
server_key: "ee9e5dc9b174558e2cd47bbd92c416e2"
server_key_iv: "e3d39cb8c42c1198b2291af70110e7a6"
server_key_version: "encryptionkey@0"
I20260806 18:34:30.054052 24853 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.006s sys 0.000s
I20260806 18:34:30.056345 24868 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 18:34:30.057483 24853 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260806 18:34:30.057623 24853 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786041246094603-20232-0/minicluster-data/ts-2/data,/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786041246094603-20232-0/minicluster-data/ts-2/wal
uuid: "3c05aac2e9314c70a8229a9f3f55ba82"
format_stamp: "Formatted at 2026-08-06 18:34:30 on dist-test-slave-1xrh"
server_key: "ee9e5dc9b174558e2cd47bbd92c416e2"
server_key_iv: "e3d39cb8c42c1198b2291af70110e7a6"
server_key_version: "encryptionkey@0"
I20260806 18:34:30.057731 24853 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786041246094603-20232-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786041246094603-20232-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786041246094603-20232-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 18:34:30.077446 24853 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 18:34:30.080641 24853 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 18:34:30.080835 24853 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 18:34:30.081374 24853 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 18:34:30.082216 24853 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 18:34:30.082285 24853 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 18:34:30.082350 24853 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 18:34:30.082398 24853 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 18:34:30.091599 24853 rpc_server.cc:307] RPC server started. Bound to: 127.19.194.3:33931
I20260806 18:34:30.091620 24989 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.194.3:33931 every 8 connection(s)
I20260806 18:34:30.092144 24853 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786041246094603-20232-0/minicluster-data/ts-2/data/info.pb
I20260806 18:34:30.095857 20232 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJW69tn/build/debug/bin/kudu as pid 24853
I20260806 18:34:30.095979 20232 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786041246094603-20232-0/minicluster-data/ts-2/wal/instance
I20260806 18:34:30.096294 20232 external_mini_cluster.cc:1428] Setting key c4b477e39b5e7fa406fe5197b8ee3cc8
Aug 06 18:34:30 dist-test-slave-1xrh krb5kdc[24455](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041270, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.194.3@KRBTEST.COM for kudu/127.19.194.62@KRBTEST.COM
I20260806 18:34:30.105273 24542 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:34:30.093755 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.194.3:48055 (local address 127.19.194.62:38703)
0806 18:34:30.093898 (+ 143us) server_negotiation.cc:206] Beginning negotiation
0806 18:34:30.093903 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 18:34:30.094780 (+ 877us) server_negotiation.cc:407] Connection header received
0806 18:34:30.096085 (+ 1305us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:34:30.096088 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:34:30.096149 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:34:30.096271 (+ 122us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:34:30.097616 (+ 1345us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:34:30.098214 (+ 598us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:34:30.099405 (+ 1191us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:34:30.099626 (+ 221us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:34:30.099708 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:34:30.101906 (+ 2198us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:34:30.101909 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:34:30.101911 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 18:34:30.101933 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:34:30.103461 (+ 1528us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:34:30.103919 (+ 458us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:34:30.103922 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:34:30.103923 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:34:30.103996 (+ 73us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:34:30.104283 (+ 287us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:34:30.104286 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:34:30.104287 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:34:30.104452 (+ 165us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:34:30.104536 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0806 18:34:30.105013 (+ 477us) server_negotiation.cc:299] Negotiation successful
0806 18:34:30.105152 (+ 139us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260806 18:34:30.106065 24992 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:34:30.094139 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.194.62:38703 (local address 127.19.194.3:48055)
0806 18:34:30.094632 (+ 493us) negotiation.cc:107] Waiting for socket to connect
0806 18:34:30.094673 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0806 18:34:30.095858 (+ 1185us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:34:30.096446 (+ 588us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:34:30.096452 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:34:30.096886 (+ 434us) client_negotiation.cc:189] Negotiated authn=SASL
0806 18:34:30.097450 (+ 564us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:34:30.097471 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:34:30.098341 (+ 870us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:34:30.098345 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:34:30.099298 (+ 953us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:34:30.099308 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:34:30.099750 (+ 442us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:34:30.099753 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:34:30.099888 (+ 135us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:34:30.100375 (+ 487us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 18:34:30.100390 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0806 18:34:30.101759 (+ 1369us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 18:34:30.103564 (+ 1805us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:34:30.103568 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:34:30.103579 (+ 11us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:34:30.103829 (+ 250us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:34:30.104078 (+ 249us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:34:30.104080 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:34:30.104081 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:34:30.104203 (+ 122us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:34:30.104539 (+ 336us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 18:34:30.104544 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 18:34:30.104747 (+ 203us) client_negotiation.cc:769] Sending connection context
0806 18:34:30.104971 (+ 224us) client_negotiation.cc:240] Negotiation successful
0806 18:34:30.105225 (+ 254us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":299,"thread_start_us":119,"threads_started":1}
I20260806 18:34:30.107280 24990 heartbeater.cc:347] Connected to a master server at 127.19.194.62:38703
I20260806 18:34:30.107514 24990 heartbeater.cc:464] Registering TS with master...
I20260806 18:34:30.107930 24990 heartbeater.cc:511] Master 127.19.194.62:38703 requested a full tablet report, sending...
I20260806 18:34:30.108986 24500 ts_manager.cc:194] Registered new tserver with Master: 3c05aac2e9314c70a8229a9f3f55ba82 (127.19.194.3:33931)
I20260806 18:34:30.109733 24500 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.19.194.3@KRBTEST.COM'} at 127.19.194.3:48055
I20260806 18:34:30.110141 20232 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260806 18:34:30.119011 24542 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:34:30.112323 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42588 (local address 127.19.194.62:38703)
0806 18:34:30.112451 (+ 128us) server_negotiation.cc:206] Beginning negotiation
0806 18:34:30.112455 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 18:34:30.112667 (+ 212us) server_negotiation.cc:407] Connection header received
0806 18:34:30.112757 (+ 90us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:34:30.112759 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:34:30.112793 (+ 34us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:34:30.112860 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:34:30.113489 (+ 629us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:34:30.113955 (+ 466us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:34:30.114695 (+ 740us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:34:30.114883 (+ 188us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:34:30.114954 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:34:30.116172 (+ 1218us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:34:30.116175 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:34:30.116177 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 18:34:30.116211 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:34:30.117646 (+ 1435us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:34:30.118019 (+ 373us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:34:30.118022 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:34:30.118024 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:34:30.118094 (+ 70us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:34:30.118339 (+ 245us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:34:30.118341 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:34:30.118342 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:34:30.118543 (+ 201us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:34:30.118645 (+ 102us) server_negotiation.cc:1035] Waiting for connection context
0806 18:34:30.118825 (+ 180us) server_negotiation.cc:299] Negotiation successful
0806 18:34:30.118908 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260806 18:34:30.121766 24500 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:42588:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260806 18:34:30.124851 24500 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 18:34:30.138814 25001 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:34:30.133040 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.194.3:33931 (local address 127.0.0.1:41410)
0806 18:34:30.133465 (+ 425us) negotiation.cc:107] Waiting for socket to connect
0806 18:34:30.133508 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0806 18:34:30.133627 (+ 119us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:34:30.134483 (+ 856us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:34:30.134491 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:34:30.134515 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 18:34:30.134655 (+ 140us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:34:30.134665 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:34:30.135866 (+ 1201us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:34:30.135869 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:34:30.137230 (+ 1361us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:34:30.137241 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:34:30.138257 (+ 1016us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:34:30.138261 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:34:30.138361 (+ 100us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:34:30.138395 (+ 34us) client_negotiation.cc:769] Sending connection context
0806 18:34:30.138537 (+ 142us) client_negotiation.cc:240] Negotiation successful
0806 18:34:30.138651 (+ 114us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":272,"mutex_wait_us":19,"thread_start_us":85,"threads_started":1}
I20260806 18:34:30.138820 25003 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:34:30.133330 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41410 (local address 127.19.194.3:33931)
0806 18:34:30.133802 (+ 472us) server_negotiation.cc:206] Beginning negotiation
0806 18:34:30.133816 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0806 18:34:30.133841 (+ 25us) server_negotiation.cc:407] Connection header received
0806 18:34:30.133917 (+ 76us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:34:30.133922 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:34:30.134133 (+ 211us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:34:30.134292 (+ 159us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 18:34:30.134795 (+ 503us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:34:30.135726 (+ 931us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:34:30.137371 (+ 1645us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:34:30.138136 (+ 765us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:34:30.138253 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:34:30.138388 (+ 135us) server_negotiation.cc:1035] Waiting for connection context
0806 18:34:30.138566 (+ 178us) server_negotiation.cc:299] Negotiation successful
0806 18:34:30.138650 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":374,"thread_start_us":144,"threads_started":1}
I20260806 18:34:30.139324 25002 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:34:30.133112 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.194.1:45271 (local address 127.0.0.1:34246)
0806 18:34:30.134055 (+ 943us) negotiation.cc:107] Waiting for socket to connect
0806 18:34:30.134069 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0806 18:34:30.134160 (+ 91us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:34:30.134862 (+ 702us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:34:30.134865 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:34:30.134881 (+ 16us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 18:34:30.134978 (+ 97us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:34:30.134984 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:34:30.136579 (+ 1595us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:34:30.136585 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:34:30.137808 (+ 1223us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:34:30.137820 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:34:30.138893 (+ 1073us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:34:30.138896 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:34:30.138982 (+ 86us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:34:30.139007 (+ 25us) client_negotiation.cc:769] Sending connection context
0806 18:34:30.139062 (+ 55us) client_negotiation.cc:240] Negotiation successful
0806 18:34:30.139155 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":884,"mutex_wait_us":71,"thread_start_us":86,"threads_started":1}
I20260806 18:34:30.139505 25004 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:34:30.133762 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.194.2:34921 (local address 127.0.0.1:35144)
0806 18:34:30.134581 (+ 819us) negotiation.cc:107] Waiting for socket to connect
0806 18:34:30.134594 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0806 18:34:30.134734 (+ 140us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:34:30.135431 (+ 697us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:34:30.135434 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:34:30.135452 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 18:34:30.135539 (+ 87us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:34:30.135544 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:34:30.136611 (+ 1067us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:34:30.136615 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:34:30.137861 (+ 1246us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:34:30.137871 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:34:30.139068 (+ 1197us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:34:30.139083 (+ 15us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:34:30.139216 (+ 133us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:34:30.139237 (+ 21us) client_negotiation.cc:769] Sending connection context
0806 18:34:30.139307 (+ 70us) client_negotiation.cc:240] Negotiation successful
0806 18:34:30.139379 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":759,"mutex_wait_us":61,"thread_start_us":88,"threads_started":1}
I20260806 18:34:30.139894 25005 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:34:30.133940 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35144 (local address 127.19.194.2:34921)
0806 18:34:30.134448 (+ 508us) server_negotiation.cc:206] Beginning negotiation
0806 18:34:30.134460 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0806 18:34:30.134668 (+ 208us) server_negotiation.cc:407] Connection header received
0806 18:34:30.134885 (+ 217us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:34:30.134890 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:34:30.135090 (+ 200us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:34:30.135233 (+ 143us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 18:34:30.135668 (+ 435us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:34:30.136470 (+ 802us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:34:30.137963 (+ 1493us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:34:30.138728 (+ 765us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:34:30.138866 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:34:30.138992 (+ 126us) server_negotiation.cc:1035] Waiting for connection context
0806 18:34:30.139651 (+ 659us) server_negotiation.cc:299] Negotiation successful
0806 18:34:30.139750 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":411,"thread_start_us":152,"threads_started":1}
I20260806 18:34:30.139930 25007 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:34:30.134008 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34246 (local address 127.19.194.1:45271)
0806 18:34:30.134395 (+ 387us) server_negotiation.cc:206] Beginning negotiation
0806 18:34:30.134406 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0806 18:34:30.134427 (+ 21us) server_negotiation.cc:407] Connection header received
0806 18:34:30.134519 (+ 92us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:34:30.134524 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:34:30.134720 (+ 196us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:34:30.135121 (+ 401us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 18:34:30.135320 (+ 199us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:34:30.136405 (+ 1085us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:34:30.137954 (+ 1549us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:34:30.138728 (+ 774us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:34:30.139503 (+ 775us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:34:30.139668 (+ 165us) server_negotiation.cc:1035] Waiting for connection context
0806 18:34:30.139713 (+ 45us) server_negotiation.cc:299] Negotiation successful
0806 18:34:30.139788 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":286,"thread_start_us":98,"threads_started":1}
I20260806 18:34:30.142223 24919 tablet_service.cc:1467] Processing CreateTablet for tablet 4f19284232b846b29f8b3b444a489d8b (DEFAULT_TABLE table=test-table [id=b992b96568bd4b55932f7629cc21ca8c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 18:34:30.142253 24633 tablet_service.cc:1467] Processing CreateTablet for tablet 4f19284232b846b29f8b3b444a489d8b (DEFAULT_TABLE table=test-table [id=b992b96568bd4b55932f7629cc21ca8c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 18:34:30.142325 24773 tablet_service.cc:1467] Processing CreateTablet for tablet 4f19284232b846b29f8b3b444a489d8b (DEFAULT_TABLE table=test-table [id=b992b96568bd4b55932f7629cc21ca8c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 18:34:30.143280 24773 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4f19284232b846b29f8b3b444a489d8b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 18:34:30.143289 24919 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4f19284232b846b29f8b3b444a489d8b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 18:34:30.143283 24633 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4f19284232b846b29f8b3b444a489d8b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 18:34:30.150014 25008 tablet_bootstrap.cc:492] T 4f19284232b846b29f8b3b444a489d8b P 16113fc2ba3941e180858ae6fd9c79af: Bootstrap starting.
I20260806 18:34:30.150604 25009 tablet_bootstrap.cc:492] T 4f19284232b846b29f8b3b444a489d8b P 03b21c075a2c446dbf938ad590297391: Bootstrap starting.
I20260806 18:34:30.151011 25010 tablet_bootstrap.cc:492] T 4f19284232b846b29f8b3b444a489d8b P 3c05aac2e9314c70a8229a9f3f55ba82: Bootstrap starting.
I20260806 18:34:30.152222 25008 tablet_bootstrap.cc:654] T 4f19284232b846b29f8b3b444a489d8b P 16113fc2ba3941e180858ae6fd9c79af: Neither blocks nor log segments found. Creating new log.
I20260806 18:34:30.152747 25010 tablet_bootstrap.cc:654] T 4f19284232b846b29f8b3b444a489d8b P 3c05aac2e9314c70a8229a9f3f55ba82: Neither blocks nor log segments found. Creating new log.
I20260806 18:34:30.153290 25008 log.cc:824] T 4f19284232b846b29f8b3b444a489d8b P 16113fc2ba3941e180858ae6fd9c79af: Log is configured to *not* fsync() on all Append() calls
I20260806 18:34:30.153838 25010 log.cc:824] T 4f19284232b846b29f8b3b444a489d8b P 3c05aac2e9314c70a8229a9f3f55ba82: Log is configured to *not* fsync() on all Append() calls
I20260806 18:34:30.155475 25008 tablet_bootstrap.cc:492] T 4f19284232b846b29f8b3b444a489d8b P 16113fc2ba3941e180858ae6fd9c79af: No bootstrap required, opened a new log
I20260806 18:34:30.155674 25009 tablet_bootstrap.cc:654] T 4f19284232b846b29f8b3b444a489d8b P 03b21c075a2c446dbf938ad590297391: Neither blocks nor log segments found. Creating new log.
I20260806 18:34:30.155681 25008 ts_tablet_manager.cc:1397] T 4f19284232b846b29f8b3b444a489d8b P 16113fc2ba3941e180858ae6fd9c79af: Time spent bootstrapping tablet: real 0.006s user 0.005s sys 0.000s
I20260806 18:34:30.155820 25010 tablet_bootstrap.cc:492] T 4f19284232b846b29f8b3b444a489d8b P 3c05aac2e9314c70a8229a9f3f55ba82: No bootstrap required, opened a new log
I20260806 18:34:30.155995 25010 ts_tablet_manager.cc:1397] T 4f19284232b846b29f8b3b444a489d8b P 3c05aac2e9314c70a8229a9f3f55ba82: Time spent bootstrapping tablet: real 0.005s user 0.003s sys 0.001s
I20260806 18:34:30.156876 25009 log.cc:824] T 4f19284232b846b29f8b3b444a489d8b P 03b21c075a2c446dbf938ad590297391: Log is configured to *not* fsync() on all Append() calls
I20260806 18:34:30.158756 25009 tablet_bootstrap.cc:492] T 4f19284232b846b29f8b3b444a489d8b P 03b21c075a2c446dbf938ad590297391: No bootstrap required, opened a new log
I20260806 18:34:30.158927 25009 ts_tablet_manager.cc:1397] T 4f19284232b846b29f8b3b444a489d8b P 03b21c075a2c446dbf938ad590297391: Time spent bootstrapping tablet: real 0.008s user 0.004s sys 0.000s
I20260806 18:34:30.159003 25010 raft_consensus.cc:348] T 4f19284232b846b29f8b3b444a489d8b P 3c05aac2e9314c70a8229a9f3f55ba82 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c05aac2e9314c70a8229a9f3f55ba82" member_type: VOTER last_known_addr { host: "127.19.194.3" port: 33931 } } peers { permanent_uuid: "03b21c075a2c446dbf938ad590297391" member_type: VOTER last_known_addr { host: "127.19.194.1" port: 45271 } } peers { permanent_uuid: "16113fc2ba3941e180858ae6fd9c79af" member_type: VOTER last_known_addr { host: "127.19.194.2" port: 34921 } }
I20260806 18:34:30.159315 25010 raft_consensus.cc:374] T 4f19284232b846b29f8b3b444a489d8b P 3c05aac2e9314c70a8229a9f3f55ba82 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 18:34:30.159408 25010 raft_consensus.cc:729] T 4f19284232b846b29f8b3b444a489d8b P 3c05aac2e9314c70a8229a9f3f55ba82 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3c05aac2e9314c70a8229a9f3f55ba82, State: Initialized, Role: FOLLOWER
I20260806 18:34:30.159462 25008 raft_consensus.cc:348] T 4f19284232b846b29f8b3b444a489d8b P 16113fc2ba3941e180858ae6fd9c79af [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c05aac2e9314c70a8229a9f3f55ba82" member_type: VOTER last_known_addr { host: "127.19.194.3" port: 33931 } } peers { permanent_uuid: "03b21c075a2c446dbf938ad590297391" member_type: VOTER last_known_addr { host: "127.19.194.1" port: 45271 } } peers { permanent_uuid: "16113fc2ba3941e180858ae6fd9c79af" member_type: VOTER last_known_addr { host: "127.19.194.2" port: 34921 } }
I20260806 18:34:30.159749 25008 raft_consensus.cc:374] T 4f19284232b846b29f8b3b444a489d8b P 16113fc2ba3941e180858ae6fd9c79af [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 18:34:30.159811 25008 raft_consensus.cc:729] T 4f19284232b846b29f8b3b444a489d8b P 16113fc2ba3941e180858ae6fd9c79af [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 16113fc2ba3941e180858ae6fd9c79af, State: Initialized, Role: FOLLOWER
I20260806 18:34:30.159883 25010 consensus_queue.cc:260] T 4f19284232b846b29f8b3b444a489d8b P 3c05aac2e9314c70a8229a9f3f55ba82 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c05aac2e9314c70a8229a9f3f55ba82" member_type: VOTER last_known_addr { host: "127.19.194.3" port: 33931 } } peers { permanent_uuid: "03b21c075a2c446dbf938ad590297391" member_type: VOTER last_known_addr { host: "127.19.194.1" port: 45271 } } peers { permanent_uuid: "16113fc2ba3941e180858ae6fd9c79af" member_type: VOTER last_known_addr { host: "127.19.194.2" port: 34921 } }
I20260806 18:34:30.160254 25008 consensus_queue.cc:260] T 4f19284232b846b29f8b3b444a489d8b P 16113fc2ba3941e180858ae6fd9c79af [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c05aac2e9314c70a8229a9f3f55ba82" member_type: VOTER last_known_addr { host: "127.19.194.3" port: 33931 } } peers { permanent_uuid: "03b21c075a2c446dbf938ad590297391" member_type: VOTER last_known_addr { host: "127.19.194.1" port: 45271 } } peers { permanent_uuid: "16113fc2ba3941e180858ae6fd9c79af" member_type: VOTER last_known_addr { host: "127.19.194.2" port: 34921 } }
I20260806 18:34:30.160552 24990 heartbeater.cc:503] Master 127.19.194.62:38703 was elected leader, sending a full tablet report...
I20260806 18:34:30.160895 25010 ts_tablet_manager.cc:1428] T 4f19284232b846b29f8b3b444a489d8b P 3c05aac2e9314c70a8229a9f3f55ba82: Time spent starting tablet: real 0.005s user 0.003s sys 0.002s
I20260806 18:34:30.161283 25008 ts_tablet_manager.cc:1428] T 4f19284232b846b29f8b3b444a489d8b P 16113fc2ba3941e180858ae6fd9c79af: Time spent starting tablet: real 0.006s user 0.006s sys 0.000s
I20260806 18:34:30.161597 24848 heartbeater.cc:503] Master 127.19.194.62:38703 was elected leader, sending a full tablet report...
I20260806 18:34:30.162533 25009 raft_consensus.cc:348] T 4f19284232b846b29f8b3b444a489d8b P 03b21c075a2c446dbf938ad590297391 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c05aac2e9314c70a8229a9f3f55ba82" member_type: VOTER last_known_addr { host: "127.19.194.3" port: 33931 } } peers { permanent_uuid: "03b21c075a2c446dbf938ad590297391" member_type: VOTER last_known_addr { host: "127.19.194.1" port: 45271 } } peers { permanent_uuid: "16113fc2ba3941e180858ae6fd9c79af" member_type: VOTER last_known_addr { host: "127.19.194.2" port: 34921 } }
I20260806 18:34:30.162734 25009 raft_consensus.cc:374] T 4f19284232b846b29f8b3b444a489d8b P 03b21c075a2c446dbf938ad590297391 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 18:34:30.162770 25009 raft_consensus.cc:729] T 4f19284232b846b29f8b3b444a489d8b P 03b21c075a2c446dbf938ad590297391 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 03b21c075a2c446dbf938ad590297391, State: Initialized, Role: FOLLOWER
I20260806 18:34:30.163127 25009 consensus_queue.cc:260] T 4f19284232b846b29f8b3b444a489d8b P 03b21c075a2c446dbf938ad590297391 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c05aac2e9314c70a8229a9f3f55ba82" member_type: VOTER last_known_addr { host: "127.19.194.3" port: 33931 } } peers { permanent_uuid: "03b21c075a2c446dbf938ad590297391" member_type: VOTER last_known_addr { host: "127.19.194.1" port: 45271 } } peers { permanent_uuid: "16113fc2ba3941e180858ae6fd9c79af" member_type: VOTER last_known_addr { host: "127.19.194.2" port: 34921 } }
I20260806 18:34:30.164001 24702 heartbeater.cc:503] Master 127.19.194.62:38703 was elected leader, sending a full tablet report...
I20260806 18:34:30.164088 25009 ts_tablet_manager.cc:1428] T 4f19284232b846b29f8b3b444a489d8b P 03b21c075a2c446dbf938ad590297391: Time spent starting tablet: real 0.005s user 0.000s sys 0.006s
W20260806 18:34:30.169368 24703 tablet.cc:2369] T 4f19284232b846b29f8b3b444a489d8b P 03b21c075a2c446dbf938ad590297391: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260806 18:34:30.343113 24991 tablet.cc:2369] T 4f19284232b846b29f8b3b444a489d8b P 3c05aac2e9314c70a8229a9f3f55ba82: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 18:34:30.376858 25016 raft_consensus.cc:482] T 4f19284232b846b29f8b3b444a489d8b P 03b21c075a2c446dbf938ad590297391 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 18:34:30.377103 25016 raft_consensus.cc:504] T 4f19284232b846b29f8b3b444a489d8b P 03b21c075a2c446dbf938ad590297391 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c05aac2e9314c70a8229a9f3f55ba82" member_type: VOTER last_known_addr { host: "127.19.194.3" port: 33931 } } peers { permanent_uuid: "03b21c075a2c446dbf938ad590297391" member_type: VOTER last_known_addr { host: "127.19.194.1" port: 45271 } } peers { permanent_uuid: "16113fc2ba3941e180858ae6fd9c79af" member_type: VOTER last_known_addr { host: "127.19.194.2" port: 34921 } }
W20260806 18:34:30.378209 24849 tablet.cc:2369] T 4f19284232b846b29f8b3b444a489d8b P 16113fc2ba3941e180858ae6fd9c79af: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 18:34:30.378929 25016 leader_election.cc:288] T 4f19284232b846b29f8b3b444a489d8b P 03b21c075a2c446dbf938ad590297391 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3c05aac2e9314c70a8229a9f3f55ba82 (127.19.194.3:33931), 16113fc2ba3941e180858ae6fd9c79af (127.19.194.2:34921)
I20260806 18:34:30.382635 25005 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:34:30.378950 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.194.1:35367 (local address 127.19.194.2:34921)
0806 18:34:30.379064 (+ 114us) server_negotiation.cc:206] Beginning negotiation
0806 18:34:30.379069 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 18:34:30.379337 (+ 268us) server_negotiation.cc:407] Connection header received
0806 18:34:30.379484 (+ 147us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:34:30.379486 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:34:30.379560 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:34:30.379641 (+ 81us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 18:34:30.379966 (+ 325us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:34:30.380448 (+ 482us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:34:30.381680 (+ 1232us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:34:30.382062 (+ 382us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:34:30.382279 (+ 217us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:34:30.382342 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0806 18:34:30.382423 (+ 81us) server_negotiation.cc:299] Negotiation successful
0806 18:34:30.382504 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260806 18:34:30.382653 25019 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:34:30.378656 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.194.2:34921 (local address 127.19.194.1:35367)
0806 18:34:30.379217 (+ 561us) negotiation.cc:107] Waiting for socket to connect
0806 18:34:30.379229 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0806 18:34:30.379352 (+ 123us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:34:30.379680 (+ 328us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:34:30.379683 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:34:30.379699 (+ 16us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 18:34:30.379838 (+ 139us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:34:30.379844 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:34:30.380575 (+ 731us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:34:30.380580 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:34:30.381529 (+ 949us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:34:30.381539 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:34:30.382209 (+ 670us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:34:30.382214 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:34:30.382318 (+ 104us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:34:30.382343 (+ 25us) client_negotiation.cc:769] Sending connection context
0806 18:34:30.382409 (+ 66us) client_negotiation.cc:240] Negotiation successful
0806 18:34:30.382468 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":496,"mutex_wait_us":111,"thread_start_us":111,"threads_started":1}
I20260806 18:34:30.382725 25003 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:34:30.378872 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.194.1:36353 (local address 127.19.194.3:33931)
0806 18:34:30.379033 (+ 161us) server_negotiation.cc:206] Beginning negotiation
0806 18:34:30.379039 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 18:34:30.379288 (+ 249us) server_negotiation.cc:407] Connection header received
0806 18:34:30.379444 (+ 156us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:34:30.379449 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:34:30.379507 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:34:30.379565 (+ 58us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 18:34:30.379934 (+ 369us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:34:30.380448 (+ 514us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:34:30.381626 (+ 1178us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:34:30.382054 (+ 428us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:34:30.382225 (+ 171us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:34:30.382285 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0806 18:34:30.382536 (+ 251us) server_negotiation.cc:299] Negotiation successful
0806 18:34:30.382604 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260806 18:34:30.383098 25018 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:34:30.378643 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.194.3:33931 (local address 127.19.194.1:36353)
0806 18:34:30.379183 (+ 540us) negotiation.cc:107] Waiting for socket to connect
0806 18:34:30.379201 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0806 18:34:30.379328 (+ 127us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:34:30.379605 (+ 277us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:34:30.379608 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:34:30.379631 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 18:34:30.379818 (+ 187us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:34:30.379825 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:34:30.380597 (+ 772us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:34:30.380600 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:34:30.381500 (+ 900us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:34:30.381510 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:34:30.382278 (+ 768us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:34:30.382281 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:34:30.382402 (+ 121us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:34:30.382426 (+ 24us) client_negotiation.cc:769] Sending connection context
0806 18:34:30.382502 (+ 76us) client_negotiation.cc:240] Negotiation successful
0806 18:34:30.382568 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":301,"mutex_wait_us":24,"thread_start_us":108,"threads_started":1}
I20260806 18:34:30.383283 24795 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4f19284232b846b29f8b3b444a489d8b" candidate_uuid: "03b21c075a2c446dbf938ad590297391" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "16113fc2ba3941e180858ae6fd9c79af" is_pre_election: true
I20260806 18:34:30.383605 24795 raft_consensus.cc:2454] T 4f19284232b846b29f8b3b444a489d8b P 16113fc2ba3941e180858ae6fd9c79af [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 03b21c075a2c446dbf938ad590297391 in term 0.
I20260806 18:34:30.383775 24941 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4f19284232b846b29f8b3b444a489d8b" candidate_uuid: "03b21c075a2c446dbf938ad590297391" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3c05aac2e9314c70a8229a9f3f55ba82" is_pre_election: true
I20260806 18:34:30.384086 24941 raft_consensus.cc:2454] T 4f19284232b846b29f8b3b444a489d8b P 3c05aac2e9314c70a8229a9f3f55ba82 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 03b21c075a2c446dbf938ad590297391 in term 0.
I20260806 18:34:30.384117 24581 leader_election.cc:302] T 4f19284232b846b29f8b3b444a489d8b P 03b21c075a2c446dbf938ad590297391 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 03b21c075a2c446dbf938ad590297391, 16113fc2ba3941e180858ae6fd9c79af; no voters:
I20260806 18:34:30.384438 25016 raft_consensus.cc:2781] T 4f19284232b846b29f8b3b444a489d8b P 03b21c075a2c446dbf938ad590297391 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 18:34:30.384539 25016 raft_consensus.cc:482] T 4f19284232b846b29f8b3b444a489d8b P 03b21c075a2c446dbf938ad590297391 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 18:34:30.384588 25016 raft_consensus.cc:3037] T 4f19284232b846b29f8b3b444a489d8b P 03b21c075a2c446dbf938ad590297391 [term 0 FOLLOWER]: Advancing to term 1
I20260806 18:34:30.385625 25016 raft_consensus.cc:504] T 4f19284232b846b29f8b3b444a489d8b P 03b21c075a2c446dbf938ad590297391 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c05aac2e9314c70a8229a9f3f55ba82" member_type: VOTER last_known_addr { host: "127.19.194.3" port: 33931 } } peers { permanent_uuid: "03b21c075a2c446dbf938ad590297391" member_type: VOTER last_known_addr { host: "127.19.194.1" port: 45271 } } peers { permanent_uuid: "16113fc2ba3941e180858ae6fd9c79af" member_type: VOTER last_known_addr { host: "127.19.194.2" port: 34921 } }
I20260806 18:34:30.385993 25016 leader_election.cc:288] T 4f19284232b846b29f8b3b444a489d8b P 03b21c075a2c446dbf938ad590297391 [CANDIDATE]: Term 1 election: Requested vote from peers 3c05aac2e9314c70a8229a9f3f55ba82 (127.19.194.3:33931), 16113fc2ba3941e180858ae6fd9c79af (127.19.194.2:34921)
I20260806 18:34:30.386273 24941 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4f19284232b846b29f8b3b444a489d8b" candidate_uuid: "03b21c075a2c446dbf938ad590297391" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3c05aac2e9314c70a8229a9f3f55ba82"
I20260806 18:34:30.386430 24795 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4f19284232b846b29f8b3b444a489d8b" candidate_uuid: "03b21c075a2c446dbf938ad590297391" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "16113fc2ba3941e180858ae6fd9c79af"
I20260806 18:34:30.386415 24941 raft_consensus.cc:3037] T 4f19284232b846b29f8b3b444a489d8b P 3c05aac2e9314c70a8229a9f3f55ba82 [term 0 FOLLOWER]: Advancing to term 1
I20260806 18:34:30.386543 24795 raft_consensus.cc:3037] T 4f19284232b846b29f8b3b444a489d8b P 16113fc2ba3941e180858ae6fd9c79af [term 0 FOLLOWER]: Advancing to term 1
I20260806 18:34:30.387857 24795 raft_consensus.cc:2454] T 4f19284232b846b29f8b3b444a489d8b P 16113fc2ba3941e180858ae6fd9c79af [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 03b21c075a2c446dbf938ad590297391 in term 1.
I20260806 18:34:30.387857 24941 raft_consensus.cc:2454] T 4f19284232b846b29f8b3b444a489d8b P 3c05aac2e9314c70a8229a9f3f55ba82 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 03b21c075a2c446dbf938ad590297391 in term 1.
I20260806 18:34:30.388149 24581 leader_election.cc:302] T 4f19284232b846b29f8b3b444a489d8b P 03b21c075a2c446dbf938ad590297391 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 03b21c075a2c446dbf938ad590297391, 16113fc2ba3941e180858ae6fd9c79af; no voters:
I20260806 18:34:30.388374 25016 raft_consensus.cc:2781] T 4f19284232b846b29f8b3b444a489d8b P 03b21c075a2c446dbf938ad590297391 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 18:34:30.388616 25016 raft_consensus.cc:686] T 4f19284232b846b29f8b3b444a489d8b P 03b21c075a2c446dbf938ad590297391 [term 1 LEADER]: Becoming Leader. State: Replica: 03b21c075a2c446dbf938ad590297391, State: Running, Role: LEADER
I20260806 18:34:30.388926 25016 consensus_queue.cc:237] T 4f19284232b846b29f8b3b444a489d8b P 03b21c075a2c446dbf938ad590297391 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c05aac2e9314c70a8229a9f3f55ba82" member_type: VOTER last_known_addr { host: "127.19.194.3" port: 33931 } } peers { permanent_uuid: "03b21c075a2c446dbf938ad590297391" member_type: VOTER last_known_addr { host: "127.19.194.1" port: 45271 } } peers { permanent_uuid: "16113fc2ba3941e180858ae6fd9c79af" member_type: VOTER last_known_addr { host: "127.19.194.2" port: 34921 } }
I20260806 18:34:30.391939 24498 catalog_manager.cc:5495] T 4f19284232b846b29f8b3b444a489d8b P 03b21c075a2c446dbf938ad590297391 reported cstate change: term changed from 0 to 1, leader changed from <none> to 03b21c075a2c446dbf938ad590297391 (127.19.194.1). New cstate: current_term: 1 leader_uuid: "03b21c075a2c446dbf938ad590297391" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c05aac2e9314c70a8229a9f3f55ba82" member_type: VOTER last_known_addr { host: "127.19.194.3" port: 33931 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "03b21c075a2c446dbf938ad590297391" member_type: VOTER last_known_addr { host: "127.19.194.1" port: 45271 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "16113fc2ba3941e180858ae6fd9c79af" member_type: VOTER last_known_addr { host: "127.19.194.2" port: 34921 } health_report { overall_health: UNKNOWN } } }
I20260806 18:34:30.428802 25007 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:34:30.425216 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34252 (local address 127.19.194.1:45271)
0806 18:34:30.425328 (+ 112us) server_negotiation.cc:206] Beginning negotiation
0806 18:34:30.425332 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 18:34:30.425345 (+ 13us) server_negotiation.cc:407] Connection header received
0806 18:34:30.425480 (+ 135us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:34:30.425483 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:34:30.425545 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:34:30.425629 (+ 84us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 18:34:30.425919 (+ 290us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:34:30.426477 (+ 558us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:34:30.427402 (+ 925us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:34:30.427671 (+ 269us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:34:30.427750 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:34:30.428108 (+ 358us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 18:34:30.428486 (+ 378us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 18:34:30.428572 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0806 18:34:30.428630 (+ 58us) server_negotiation.cc:299] Negotiation successful
0806 18:34:30.428696 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":33}
I20260806 18:34:30.434489 24941 raft_consensus.cc:1261] T 4f19284232b846b29f8b3b444a489d8b P 3c05aac2e9314c70a8229a9f3f55ba82 [term 1 FOLLOWER]: Refusing update from remote peer 03b21c075a2c446dbf938ad590297391: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 18:34:30.434478 24795 raft_consensus.cc:1261] T 4f19284232b846b29f8b3b444a489d8b P 16113fc2ba3941e180858ae6fd9c79af [term 1 FOLLOWER]: Refusing update from remote peer 03b21c075a2c446dbf938ad590297391: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 18:34:30.435251 25016 consensus_queue.cc:1035] T 4f19284232b846b29f8b3b444a489d8b P 03b21c075a2c446dbf938ad590297391 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3c05aac2e9314c70a8229a9f3f55ba82" member_type: VOTER last_known_addr { host: "127.19.194.3" port: 33931 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 18:34:30.435506 25020 consensus_queue.cc:1035] T 4f19284232b846b29f8b3b444a489d8b P 03b21c075a2c446dbf938ad590297391 [LEADER]: Connected to new peer: Peer: permanent_uuid: "16113fc2ba3941e180858ae6fd9c79af" member_type: VOTER last_known_addr { host: "127.19.194.2" port: 34921 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 18:34:30.442873 25023 mvcc.cc:205] Tried to move back new op lower bound from 7315625043692097536 to 7315625043515338752. Current Snapshot: MvccSnapshot[applied={T|T < 7315625043692097536}]
I20260806 18:34:30.444837 25024 mvcc.cc:205] Tried to move back new op lower bound from 7315625043692097536 to 7315625043515338752. Current Snapshot: MvccSnapshot[applied={T|T < 7315625043692097536}]
I20260806 18:34:30.445024 25025 mvcc.cc:205] Tried to move back new op lower bound from 7315625043692097536 to 7315625043515338752. Current Snapshot: MvccSnapshot[applied={T|T < 7315625043692097536}]
I20260806 18:34:30.451732 24499 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:42588:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 18:34:30.451910 24499 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:42588:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 18:34:30.455054 24499 catalog_manager.cc:5782] T 00000000000000000000000000000000 P eca707e6ea054f71a1aaaa6463831932: Sending DeleteTablet for 3 replicas of tablet 4f19284232b846b29f8b3b444a489d8b
I20260806 18:34:30.455802 24919 tablet_service.cc:1514] Processing DeleteTablet for tablet 4f19284232b846b29f8b3b444a489d8b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 18:34:30 UTC) from {username='kudu', principal='kudu/127.19.194.62@KRBTEST.COM'} at 127.0.0.1:41410
I20260806 18:34:30.456295 25032 tablet_replica.cc:330] T 4f19284232b846b29f8b3b444a489d8b P 3c05aac2e9314c70a8229a9f3f55ba82: stopping tablet replica
I20260806 18:34:30.456316 24773 tablet_service.cc:1514] Processing DeleteTablet for tablet 4f19284232b846b29f8b3b444a489d8b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 18:34:30 UTC) from {username='kudu', principal='kudu/127.19.194.62@KRBTEST.COM'} at 127.0.0.1:35144
I20260806 18:34:30.456622 25032 raft_consensus.cc:2229] T 4f19284232b846b29f8b3b444a489d8b P 3c05aac2e9314c70a8229a9f3f55ba82 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 18:34:30.456784 25033 tablet_replica.cc:330] T 4f19284232b846b29f8b3b444a489d8b P 16113fc2ba3941e180858ae6fd9c79af: stopping tablet replica
I20260806 18:34:30.456943 25032 raft_consensus.cc:2258] T 4f19284232b846b29f8b3b444a489d8b P 3c05aac2e9314c70a8229a9f3f55ba82 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 18:34:30.457100 25033 raft_consensus.cc:2229] T 4f19284232b846b29f8b3b444a489d8b P 16113fc2ba3941e180858ae6fd9c79af [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 18:34:30.457403 25033 raft_consensus.cc:2258] T 4f19284232b846b29f8b3b444a489d8b P 16113fc2ba3941e180858ae6fd9c79af [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 18:34:30.458583 24633 tablet_service.cc:1514] Processing DeleteTablet for tablet 4f19284232b846b29f8b3b444a489d8b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 18:34:30 UTC) from {username='kudu', principal='kudu/127.19.194.62@KRBTEST.COM'} at 127.0.0.1:34246
I20260806 18:34:30.458869 25033 ts_tablet_manager.cc:1905] T 4f19284232b846b29f8b3b444a489d8b P 16113fc2ba3941e180858ae6fd9c79af: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 18:34:30.460413 20232 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJW69tn/build/debug/bin/kudu with pid 24566
I20260806 18:34:30.462113 25032 ts_tablet_manager.cc:1905] T 4f19284232b846b29f8b3b444a489d8b P 3c05aac2e9314c70a8229a9f3f55ba82: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 18:34:30.462574 25033 ts_tablet_manager.cc:1918] T 4f19284232b846b29f8b3b444a489d8b P 16113fc2ba3941e180858ae6fd9c79af: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 18:34:30.462704 25033 log.cc:1196] T 4f19284232b846b29f8b3b444a489d8b P 16113fc2ba3941e180858ae6fd9c79af: Deleting WAL directory at /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786041246094603-20232-0/minicluster-data/ts-1/wal/wals/4f19284232b846b29f8b3b444a489d8b
I20260806 18:34:30.463150 25033 ts_tablet_manager.cc:1939] T 4f19284232b846b29f8b3b444a489d8b P 16113fc2ba3941e180858ae6fd9c79af: Deleting consensus metadata
I20260806 18:34:30.464392 24484 catalog_manager.cc:4841] TS 16113fc2ba3941e180858ae6fd9c79af (127.19.194.2:34921): tablet 4f19284232b846b29f8b3b444a489d8b (table test-table [id=b992b96568bd4b55932f7629cc21ca8c]) successfully deleted
I20260806 18:34:30.464957 25032 ts_tablet_manager.cc:1918] T 4f19284232b846b29f8b3b444a489d8b P 3c05aac2e9314c70a8229a9f3f55ba82: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 18:34:30.465080 25032 log.cc:1196] T 4f19284232b846b29f8b3b444a489d8b P 3c05aac2e9314c70a8229a9f3f55ba82: Deleting WAL directory at /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786041246094603-20232-0/minicluster-data/ts-2/wal/wals/4f19284232b846b29f8b3b444a489d8b
I20260806 18:34:30.465394 25032 ts_tablet_manager.cc:1939] T 4f19284232b846b29f8b3b444a489d8b P 3c05aac2e9314c70a8229a9f3f55ba82: Deleting consensus metadata
I20260806 18:34:30.466341 24487 catalog_manager.cc:4841] TS 3c05aac2e9314c70a8229a9f3f55ba82 (127.19.194.3:33931): tablet 4f19284232b846b29f8b3b444a489d8b (table test-table [id=b992b96568bd4b55932f7629cc21ca8c]) successfully deleted
W20260806 18:34:30.471410 24484 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.19.194.1:45271 (error 108)
I20260806 18:34:30.471678 20232 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJW69tn/build/debug/bin/kudu with pid 24708
I20260806 18:34:30.472959 25004 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:34:30.472522 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.194.1:45271 (local address 127.0.0.1:34268)
0806 18:34:30.472747 (+ 225us) negotiation.cc:107] Waiting for socket to connect
0806 18:34:30.472825 (+ 78us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.194.1:45271: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":127}
W20260806 18:34:30.473531 24484 catalog_manager.cc:4584] TS 03b21c075a2c446dbf938ad590297391 (127.19.194.1:45271): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 4f19284232b846b29f8b3b444a489d8b: Network error: Client connection negotiation failed: client connection to 127.19.194.1:45271: connect: Connection refused (error 111)
I20260806 18:34:30.477998 20232 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJW69tn/build/debug/bin/kudu with pid 24853
I20260806 18:34:30.484798 20232 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJW69tn/build/debug/bin/kudu with pid 24469
2026-08-06T18:34:30Z chronyd exiting
[ OK ] SecurityITest.TestDisableWebUI (2447 ms)
[ RUN ] SecurityITest.TestDisableAuthenticationEncryption
2026-08-06T18:34:30Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T18:34:30Z Disabled control of system clock
I20260806 18:34:30.515710 20232 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJW69tn/build/debug/bin/kudu
/tmp/dist-test-taskJW69tn/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786041246094603-20232-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786041246094603-20232-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786041246094603-20232-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786041246094603-20232-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.19.194.62:42911
--webserver_interface=127.19.194.62
--webserver_port=0
--builtin_ntp_servers=127.19.194.20:46195
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.19.194.62:42911
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
W20260806 18:34:30.624661 25039 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 18:34:30.624907 25039 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 18:34:30.624943 25039 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 18:34:30.628842 25039 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 18:34:30.628893 25039 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 18:34:30.628908 25039 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 18:34:30.628924 25039 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 18:34:30.628939 25039 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 18:34:30.633558 25039 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.194.20:46195
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786041246094603-20232-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786041246094603-20232-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.19.194.62:42911
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.19.194.62:42911
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786041246094603-20232-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.19.194.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25039
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786041246094603-20232-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 17:59:13 UTC on ccycloud.quasar-wvsnzx.root.comops.site
build id 4571028
I20260806 18:34:30.634482 25039 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 18:34:30.635810 25039 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 18:34:30.643821 25045 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:34:30.643895 25047 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 18:34:30.643975 25039 server_base.cc:1034] running on GCE node
W20260806 18:34:30.643816 25044 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 18:34:30.644596 25039 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 18:34:30.645618 25039 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 18:34:30.646807 25039 hybrid_clock.cc:648] HybridClock initialized: now 1786041270646782 us; error 36 us; skew 500 ppm
I20260806 18:34:30.648696 25039 webserver.cc:460] Webserver started at http://127.19.194.62:33499/ using document root <none> and password file <none>
I20260806 18:34:30.649258 25039 fs_manager.cc:359] Metadata directory not provided
I20260806 18:34:30.649353 25039 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 18:34:30.649592 25039 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 18:34:30.651623 25039 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786041246094603-20232-0/minicluster-data/master-0/data/instance:
uuid: "901fc9604d1d4e94928432c374cf7f44"
format_stamp: "Formatted at 2026-08-06 18:34:30 on dist-test-slave-1xrh"
server_key: "79d04943267ab5e8814114c5c482f3fc"
server_key_iv: "7da8841ec175d142e63e2347f48e2723"
server_key_version: "encryptionkey@0"
I20260806 18:34:30.652186 25039 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786041246094603-20232-0/minicluster-data/master-0/wal/instance:
uuid: "901fc9604d1d4e94928432c374cf7f44"
format_stamp: "Formatted at 2026-08-06 18:34:30 on dist-test-slave-1xrh"
server_key: "79d04943267ab5e8814114c5c482f3fc"
server_key_iv: "7da8841ec175d142e63e2347f48e2723"
server_key_version: "encryptionkey@0"
I20260806 18:34:30.655782 25039 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.001s
I20260806 18:34:30.658185 25053 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 18:34:30.659386 25039 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.001s
I20260806 18:34:30.659519 25039 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786041246094603-20232-0/minicluster-data/master-0/data,/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786041246094603-20232-0/minicluster-data/master-0/wal
uuid: "901fc9604d1d4e94928432c374cf7f44"
format_stamp: "Formatted at 2026-08-06 18:34:30 on dist-test-slave-1xrh"
server_key: "79d04943267ab5e8814114c5c482f3fc"
server_key_iv: "7da8841ec175d142e63e2347f48e2723"
server_key_version: "encryptionkey@0"
I20260806 18:34:30.659638 25039 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786041246094603-20232-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786041246094603-20232-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786041246094603-20232-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 18:34:30.663998 25039 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 18:34:30.664577 25039 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 18:34:30.664754 25039 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 18:34:30.671305 25039 rpc_server.cc:307] RPC server started. Bound to: 127.19.194.62:42911
I20260806 18:34:30.671317 25108 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.194.62:42911 every 8 connection(s)
I20260806 18:34:30.672163 25039 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786041246094603-20232-0/minicluster-data/master-0/data/info.pb
I20260806 18:34:30.675076 25109 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 18:34:30.680217 25109 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 901fc9604d1d4e94928432c374cf7f44: Bootstrap starting.
I20260806 18:34:30.681169 20232 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJW69tn/build/debug/bin/kudu as pid 25039
I20260806 18:34:30.681298 20232 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786041246094603-20232-0/minicluster-data/master-0/wal/instance
I20260806 18:34:30.681617 20232 external_mini_cluster.cc:1428] Setting key 53fa63690c509fc2ab6b3eefeea8d9d6
I20260806 18:34:30.683034 25109 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 901fc9604d1d4e94928432c374cf7f44: Neither blocks nor log segments found. Creating new log.
I20260806 18:34:30.683755 25109 log.cc:824] T 00000000000000000000000000000000 P 901fc9604d1d4e94928432c374cf7f44: Log is configured to *not* fsync() on all Append() calls
I20260806 18:34:30.685703 25109 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 901fc9604d1d4e94928432c374cf7f44: No bootstrap required, opened a new log
I20260806 18:34:30.686784 25112 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:34:30.682773 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51880 (local address 127.19.194.62:42911)
0806 18:34:30.683476 (+ 703us) server_negotiation.cc:206] Beginning negotiation
0806 18:34:30.683492 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0806 18:34:30.683527 (+ 35us) server_negotiation.cc:407] Connection header received
0806 18:34:30.684198 (+ 671us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:34:30.684216 (+ 18us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:34:30.684483 (+ 267us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:34:30.684804 (+ 321us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:34:30.685592 (+ 788us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:34:30.685597 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:34:30.685608 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 18:34:30.685619 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:34:30.685671 (+ 52us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 18:34:30.685687 (+ 16us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:34:30.685806 (+ 119us) server_negotiation.cc:1035] Waiting for connection context
0806 18:34:30.685911 (+ 105us) server_negotiation.cc:299] Negotiation successful
0806 18:34:30.686154 (+ 243us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":353,"thread_start_us":149,"threads_started":1}
I20260806 18:34:30.689709 25109 raft_consensus.cc:348] T 00000000000000000000000000000000 P 901fc9604d1d4e94928432c374cf7f44 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "901fc9604d1d4e94928432c374cf7f44" member_type: VOTER last_known_addr { host: "127.19.194.62" port: 42911 } }
I20260806 18:34:30.689932 25109 raft_consensus.cc:374] T 00000000000000000000000000000000 P 901fc9604d1d4e94928432c374cf7f44 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 18:34:30.690011 25109 raft_consensus.cc:729] T 00000000000000000000000000000000 P 901fc9604d1d4e94928432c374cf7f44 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 901fc9604d1d4e94928432c374cf7f44, State: Initialized, Role: FOLLOWER
I20260806 18:34:30.690446 25109 consensus_queue.cc:260] T 00000000000000000000000000000000 P 901fc9604d1d4e94928432c374cf7f44 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "901fc9604d1d4e94928432c374cf7f44" member_type: VOTER last_known_addr { host: "127.19.194.62" port: 42911 } }
I20260806 18:34:30.690596 25109 raft_consensus.cc:388] T 00000000000000000000000000000000 P 901fc9604d1d4e94928432c374cf7f44 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 18:34:30.690678 25109 raft_consensus.cc:482] T 00000000000000000000000000000000 P 901fc9604d1d4e94928432c374cf7f44 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 18:34:30.690779 25109 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 901fc9604d1d4e94928432c374cf7f44 [term 0 FOLLOWER]: Advancing to term 1
I20260806 18:34:30.691711 25109 raft_consensus.cc:504] T 00000000000000000000000000000000 P 901fc9604d1d4e94928432c374cf7f44 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "901fc9604d1d4e94928432c374cf7f44" member_type: VOTER last_known_addr { host: "127.19.194.62" port: 42911 } }
I20260806 18:34:30.692027 25109 leader_election.cc:302] T 00000000000000000000000000000000 P 901fc9604d1d4e94928432c374cf7f44 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 901fc9604d1d4e94928432c374cf7f44; no voters:
I20260806 18:34:30.692392 25109 leader_election.cc:288] T 00000000000000000000000000000000 P 901fc9604d1d4e94928432c374cf7f44 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 18:34:30.692498 25114 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 901fc9604d1d4e94928432c374cf7f44 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 18:34:30.692741 25114 raft_consensus.cc:686] T 00000000000000000000000000000000 P 901fc9604d1d4e94928432c374cf7f44 [term 1 LEADER]: Becoming Leader. State: Replica: 901fc9604d1d4e94928432c374cf7f44, State: Running, Role: LEADER
I20260806 18:34:30.693041 25114 consensus_queue.cc:237] T 00000000000000000000000000000000 P 901fc9604d1d4e94928432c374cf7f44 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "901fc9604d1d4e94928432c374cf7f44" member_type: VOTER last_known_addr { host: "127.19.194.62" port: 42911 } }
I20260806 18:34:30.693585 25109 sys_catalog.cc:564] T 00000000000000000000000000000000 P 901fc9604d1d4e94928432c374cf7f44 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 18:34:30.694697 25115 sys_catalog.cc:455] T 00000000000000000000000000000000 P 901fc9604d1d4e94928432c374cf7f44 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "901fc9604d1d4e94928432c374cf7f44" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "901fc9604d1d4e94928432c374cf7f44" member_type: VOTER last_known_addr { host: "127.19.194.62" port: 42911 } } }
I20260806 18:34:30.694829 25115 sys_catalog.cc:458] T 00000000000000000000000000000000 P 901fc9604d1d4e94928432c374cf7f44 [sys.catalog]: This master's current role is: LEADER
I20260806 18:34:30.695070 25116 sys_catalog.cc:455] T 00000000000000000000000000000000 P 901fc9604d1d4e94928432c374cf7f44 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 901fc9604d1d4e94928432c374cf7f44. Latest consensus state: current_term: 1 leader_uuid: "901fc9604d1d4e94928432c374cf7f44" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "901fc9604d1d4e94928432c374cf7f44" member_type: VOTER last_known_addr { host: "127.19.194.62" port: 42911 } } }
I20260806 18:34:30.695197 25116 sys_catalog.cc:458] T 00000000000000000000000000000000 P 901fc9604d1d4e94928432c374cf7f44 [sys.catalog]: This master's current role is: LEADER
I20260806 18:34:30.696180 25123 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 18:34:30.698597 25123 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 18:34:30.703590 25123 catalog_manager.cc:1347] Generated new cluster ID: 0c619c88141b432086ab4b4a361d8b1f
I20260806 18:34:30.703652 25123 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 18:34:30.731915 25123 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 18:34:30.732220 25123 catalog_manager.cc:1504] Loading token signing keys...
I20260806 18:34:30.737336 25123 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 901fc9604d1d4e94928432c374cf7f44: Generated new TSK 0
I20260806 18:34:30.738063 25123 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 18:34:30.744663 20232 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJW69tn/build/debug/bin/kudu
/tmp/dist-test-taskJW69tn/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786041246094603-20232-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786041246094603-20232-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786041246094603-20232-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786041246094603-20232-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.19.194.1:0
--local_ip_for_outbound_sockets=127.19.194.1
--webserver_interface=127.19.194.1
--webserver_port=0
--tserver_master_addrs=127.19.194.62:42911
--builtin_ntp_servers=127.19.194.20:46195
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
W20260806 18:34:30.871515 25133 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 18:34:30.871765 25133 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 18:34:30.871829 25133 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 18:34:30.875828 25133 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 18:34:30.875896 25133 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 18:34:30.875988 25133 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.19.194.1
I20260806 18:34:30.880570 25133 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.194.20:46195
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786041246094603-20232-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786041246094603-20232-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.19.194.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786041246094603-20232-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.19.194.1
--webserver_port=0
--tserver_master_addrs=127.19.194.62:42911
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25133
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.19.194.1
--log_dir=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786041246094603-20232-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 17:59:13 UTC on ccycloud.quasar-wvsnzx.root.comops.site
build id 4571028
I20260806 18:34:30.881593 25133 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 18:34:30.882957 25133 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 18:34:30.891993 25138 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:34:30.892057 25142 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:34:30.891993 25140 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 18:34:30.892298 25133 server_base.cc:1034] running on GCE node
I20260806 18:34:30.892660 25133 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 18:34:30.893272 25133 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 18:34:30.894449 25133 hybrid_clock.cc:648] HybridClock initialized: now 1786041270894422 us; error 43 us; skew 500 ppm
I20260806 18:34:30.896370 25133 webserver.cc:460] Webserver started at http://127.19.194.1:32969/ using document root <none> and password file <none>
I20260806 18:34:30.896903 25133 fs_manager.cc:359] Metadata directory not provided
I20260806 18:34:30.896992 25133 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 18:34:30.897215 25133 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 18:34:30.899119 25133 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786041246094603-20232-0/minicluster-data/ts-0/data/instance:
uuid: "24f7daf40ef341f9b96a4c8ddcbe9fde"
format_stamp: "Formatted at 2026-08-06 18:34:30 on dist-test-slave-1xrh"
server_key: "110b4a5de43557caaab42a5d96d9b492"
server_key_iv: "2d98ca3655aaba3f2bc16987abdbcf8c"
server_key_version: "encryptionkey@0"
I20260806 18:34:30.899736 25133 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786041246094603-20232-0/minicluster-data/ts-0/wal/instance:
uuid: "24f7daf40ef341f9b96a4c8ddcbe9fde"
format_stamp: "Formatted at 2026-08-06 18:34:30 on dist-test-slave-1xrh"
server_key: "110b4a5de43557caaab42a5d96d9b492"
server_key_iv: "2d98ca3655aaba3f2bc16987abdbcf8c"
server_key_version: "encryptionkey@0"
I20260806 18:34:30.903103 25133 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.000s
I20260806 18:34:30.905437 25148 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 18:34:30.906668 25133 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260806 18:34:30.906792 25133 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786041246094603-20232-0/minicluster-data/ts-0/data,/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786041246094603-20232-0/minicluster-data/ts-0/wal
uuid: "24f7daf40ef341f9b96a4c8ddcbe9fde"
format_stamp: "Formatted at 2026-08-06 18:34:30 on dist-test-slave-1xrh"
server_key: "110b4a5de43557caaab42a5d96d9b492"
server_key_iv: "2d98ca3655aaba3f2bc16987abdbcf8c"
server_key_version: "encryptionkey@0"
I20260806 18:34:30.906908 25133 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786041246094603-20232-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786041246094603-20232-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786041246094603-20232-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 18:34:30.911387 25133 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 18:34:30.911958 25133 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 18:34:30.912142 25133 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 18:34:30.912657 25133 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 18:34:30.913491 25133 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 18:34:30.913560 25133 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 18:34:30.913616 25133 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 18:34:30.913666 25133 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 18:34:30.922565 25133 rpc_server.cc:307] RPC server started. Bound to: 127.19.194.1:38513
I20260806 18:34:30.922591 25266 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.194.1:38513 every 8 connection(s)
I20260806 18:34:30.923444 25133 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786041246094603-20232-0/minicluster-data/ts-0/data/info.pb
I20260806 18:34:30.929294 25112 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:34:30.925216 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.194.1:57481 (local address 127.19.194.62:42911)
0806 18:34:30.925387 (+ 171us) server_negotiation.cc:206] Beginning negotiation
0806 18:34:30.925392 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 18:34:30.926368 (+ 976us) server_negotiation.cc:407] Connection header received
0806 18:34:30.927804 (+ 1436us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:34:30.927806 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:34:30.927843 (+ 37us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:34:30.927918 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:34:30.928918 (+ 1000us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:34:30.928921 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:34:30.928923 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 18:34:30.928936 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:34:30.928956 (+ 20us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 18:34:30.928965 (+ 9us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:34:30.929042 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0806 18:34:30.929157 (+ 115us) server_negotiation.cc:299] Negotiation successful
0806 18:34:30.929193 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260806 18:34:30.930065 25269 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:34:30.925607 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.194.62:42911 (local address 127.19.194.1:57481)
0806 18:34:30.926226 (+ 619us) negotiation.cc:107] Waiting for socket to connect
0806 18:34:30.926265 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0806 18:34:30.927624 (+ 1359us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:34:30.928070 (+ 446us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:34:30.928076 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:34:30.928159 (+ 83us) client_negotiation.cc:189] Negotiated authn=SASL
0806 18:34:30.928670 (+ 511us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0806 18:34:30.928672 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0806 18:34:30.928709 (+ 37us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0806 18:34:30.928775 (+ 66us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 18:34:30.929040 (+ 265us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 18:34:30.929047 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 18:34:30.929052 (+ 5us) client_negotiation.cc:769] Sending connection context
0806 18:34:30.929173 (+ 121us) client_negotiation.cc:240] Negotiation successful
0806 18:34:30.929347 (+ 174us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":245,"thread_start_us":108,"threads_started":1}
I20260806 18:34:30.931208 25267 heartbeater.cc:347] Connected to a master server at 127.19.194.62:42911
I20260806 18:34:30.931424 25267 heartbeater.cc:464] Registering TS with master...
I20260806 18:34:30.931876 20232 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJW69tn/build/debug/bin/kudu as pid 25133
I20260806 18:34:30.931876 25267 heartbeater.cc:511] Master 127.19.194.62:42911 requested a full tablet report, sending...
I20260806 18:34:30.931989 20232 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786041246094603-20232-0/minicluster-data/ts-0/wal/instance
I20260806 18:34:30.932309 20232 external_mini_cluster.cc:1428] Setting key 3b216077ce1f7de0809e0077bcf39eb8
I20260806 18:34:30.933451 25070 ts_manager.cc:194] Registered new tserver with Master: 24f7daf40ef341f9b96a4c8ddcbe9fde (127.19.194.1:38513)
I20260806 18:34:30.934271 20232 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJW69tn/build/debug/bin/kudu
/tmp/dist-test-taskJW69tn/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786041246094603-20232-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786041246094603-20232-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786041246094603-20232-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786041246094603-20232-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.19.194.2:0
--local_ip_for_outbound_sockets=127.19.194.2
--webserver_interface=127.19.194.2
--webserver_port=0
--tserver_master_addrs=127.19.194.62:42911
--builtin_ntp_servers=127.19.194.20:46195
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
W20260806 18:34:31.044528 25270 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 18:34:31.044761 25270 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 18:34:31.044799 25270 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 18:34:31.048686 25270 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 18:34:31.048727 25270 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 18:34:31.048775 25270 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.19.194.2
I20260806 18:34:31.053269 25270 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.194.20:46195
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786041246094603-20232-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786041246094603-20232-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.19.194.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786041246094603-20232-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.19.194.2
--webserver_port=0
--tserver_master_addrs=127.19.194.62:42911
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25270
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.19.194.2
--log_dir=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786041246094603-20232-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 17:59:13 UTC on ccycloud.quasar-wvsnzx.root.comops.site
build id 4571028
I20260806 18:34:31.054208 25270 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 18:34:31.055495 25270 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 18:34:31.064841 25278 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:34:31.064891 25275 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:34:31.064847 25276 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 18:34:31.065052 25270 server_base.cc:1034] running on GCE node
I20260806 18:34:31.065522 25270 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 18:34:31.066190 25270 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 18:34:31.067376 25270 hybrid_clock.cc:648] HybridClock initialized: now 1786041271067360 us; error 37 us; skew 500 ppm
I20260806 18:34:31.069147 25270 webserver.cc:460] Webserver started at http://127.19.194.2:38153/ using document root <none> and password file <none>
I20260806 18:34:31.069710 25270 fs_manager.cc:359] Metadata directory not provided
I20260806 18:34:31.069803 25270 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 18:34:31.070020 25270 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 18:34:31.072082 25270 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786041246094603-20232-0/minicluster-data/ts-1/data/instance:
uuid: "3557d45d2daa4d65b3cfe14511b486e6"
format_stamp: "Formatted at 2026-08-06 18:34:31 on dist-test-slave-1xrh"
server_key: "538e4c3e1a85b2ce1456e28aaecc7774"
server_key_iv: "3541916798b2b330c0410fe50f1ac7dc"
server_key_version: "encryptionkey@0"
I20260806 18:34:31.072638 25270 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786041246094603-20232-0/minicluster-data/ts-1/wal/instance:
uuid: "3557d45d2daa4d65b3cfe14511b486e6"
format_stamp: "Formatted at 2026-08-06 18:34:31 on dist-test-slave-1xrh"
server_key: "538e4c3e1a85b2ce1456e28aaecc7774"
server_key_iv: "3541916798b2b330c0410fe50f1ac7dc"
server_key_version: "encryptionkey@0"
I20260806 18:34:31.076257 25270 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20260806 18:34:31.078756 25284 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 18:34:31.079945 25270 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260806 18:34:31.080096 25270 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786041246094603-20232-0/minicluster-data/ts-1/data,/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786041246094603-20232-0/minicluster-data/ts-1/wal
uuid: "3557d45d2daa4d65b3cfe14511b486e6"
format_stamp: "Formatted at 2026-08-06 18:34:31 on dist-test-slave-1xrh"
server_key: "538e4c3e1a85b2ce1456e28aaecc7774"
server_key_iv: "3541916798b2b330c0410fe50f1ac7dc"
server_key_version: "encryptionkey@0"
I20260806 18:34:31.080214 25270 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786041246094603-20232-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786041246094603-20232-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786041246094603-20232-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 18:34:31.084645 25270 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 18:34:31.085235 25270 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 18:34:31.085417 25270 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 18:34:31.085933 25270 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 18:34:31.086809 25270 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 18:34:31.086899 25270 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 18:34:31.086967 25270 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 18:34:31.087013 25270 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 18:34:31.096032 25270 rpc_server.cc:307] RPC server started. Bound to: 127.19.194.2:34431
I20260806 18:34:31.096063 25403 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.194.2:34431 every 8 connection(s)
I20260806 18:34:31.096890 25270 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786041246094603-20232-0/minicluster-data/ts-1/data/info.pb
I20260806 18:34:31.100602 20232 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJW69tn/build/debug/bin/kudu as pid 25270
I20260806 18:34:31.100752 20232 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786041246094603-20232-0/minicluster-data/ts-1/wal/instance
I20260806 18:34:31.101089 20232 external_mini_cluster.cc:1428] Setting key 79a4661430af98e43e7cc8a084e65d5e
I20260806 18:34:31.103047 25112 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:34:31.098598 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.194.2:49019 (local address 127.19.194.62:42911)
0806 18:34:31.098746 (+ 148us) server_negotiation.cc:206] Beginning negotiation
0806 18:34:31.098751 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 18:34:31.099820 (+ 1069us) server_negotiation.cc:407] Connection header received
0806 18:34:31.101367 (+ 1547us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:34:31.101370 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:34:31.101409 (+ 39us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:34:31.101510 (+ 101us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:34:31.102573 (+ 1063us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:34:31.102577 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:34:31.102580 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 18:34:31.102592 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:34:31.102615 (+ 23us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 18:34:31.102630 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:34:31.102722 (+ 92us) server_negotiation.cc:1035] Waiting for connection context
0806 18:34:31.102871 (+ 149us) server_negotiation.cc:299] Negotiation successful
0806 18:34:31.102918 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260806 18:34:31.103293 20232 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJW69tn/build/debug/bin/kudu
/tmp/dist-test-taskJW69tn/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786041246094603-20232-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786041246094603-20232-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786041246094603-20232-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786041246094603-20232-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.19.194.3:0
--local_ip_for_outbound_sockets=127.19.194.3
--webserver_interface=127.19.194.3
--webserver_port=0
--tserver_master_addrs=127.19.194.62:42911
--builtin_ntp_servers=127.19.194.20:46195
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
I20260806 18:34:31.103964 25406 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:34:31.099006 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.194.62:42911 (local address 127.19.194.2:49019)
0806 18:34:31.099673 (+ 667us) negotiation.cc:107] Waiting for socket to connect
0806 18:34:31.099711 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0806 18:34:31.101152 (+ 1441us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:34:31.101639 (+ 487us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:34:31.101645 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:34:31.101727 (+ 82us) client_negotiation.cc:189] Negotiated authn=SASL
0806 18:34:31.102327 (+ 600us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0806 18:34:31.102329 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0806 18:34:31.102365 (+ 36us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0806 18:34:31.102428 (+ 63us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 18:34:31.102711 (+ 283us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 18:34:31.102721 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 18:34:31.102727 (+ 6us) client_negotiation.cc:769] Sending connection context
0806 18:34:31.102901 (+ 174us) client_negotiation.cc:240] Negotiation successful
0806 18:34:31.103164 (+ 263us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":295,"thread_start_us":157,"threads_started":1}
I20260806 18:34:31.105031 25404 heartbeater.cc:347] Connected to a master server at 127.19.194.62:42911
I20260806 18:34:31.105264 25404 heartbeater.cc:464] Registering TS with master...
I20260806 18:34:31.105639 25404 heartbeater.cc:511] Master 127.19.194.62:42911 requested a full tablet report, sending...
I20260806 18:34:31.106639 25070 ts_manager.cc:194] Registered new tserver with Master: 3557d45d2daa4d65b3cfe14511b486e6 (127.19.194.2:34431)
W20260806 18:34:31.215102 25407 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 18:34:31.215421 25407 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 18:34:31.215459 25407 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 18:34:31.219314 25407 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 18:34:31.219357 25407 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 18:34:31.219408 25407 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.19.194.3
I20260806 18:34:31.223817 25407 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.194.20:46195
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786041246094603-20232-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786041246094603-20232-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.19.194.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786041246094603-20232-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.19.194.3
--webserver_port=0
--tserver_master_addrs=127.19.194.62:42911
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25407
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.19.194.3
--log_dir=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786041246094603-20232-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 17:59:13 UTC on ccycloud.quasar-wvsnzx.root.comops.site
build id 4571028
I20260806 18:34:31.224745 25407 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 18:34:31.225986 25407 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 18:34:31.234823 25412 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:34:31.234885 25414 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:34:31.234859 25416 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 18:34:31.235176 25407 server_base.cc:1034] running on GCE node
I20260806 18:34:31.235548 25407 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 18:34:31.236160 25407 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 18:34:31.237351 25407 hybrid_clock.cc:648] HybridClock initialized: now 1786041271237330 us; error 44 us; skew 500 ppm
I20260806 18:34:31.239254 25407 webserver.cc:460] Webserver started at http://127.19.194.3:35117/ using document root <none> and password file <none>
I20260806 18:34:31.239809 25407 fs_manager.cc:359] Metadata directory not provided
I20260806 18:34:31.239905 25407 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 18:34:31.240129 25407 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 18:34:31.242174 25407 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786041246094603-20232-0/minicluster-data/ts-2/data/instance:
uuid: "4850c1053d124036921de52053cfff53"
format_stamp: "Formatted at 2026-08-06 18:34:31 on dist-test-slave-1xrh"
server_key: "f65322bb79d395b079b70a5d97cd4d1b"
server_key_iv: "a3cb595100895b0e72d16a217bd022b2"
server_key_version: "encryptionkey@0"
I20260806 18:34:31.242739 25407 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786041246094603-20232-0/minicluster-data/ts-2/wal/instance:
uuid: "4850c1053d124036921de52053cfff53"
format_stamp: "Formatted at 2026-08-06 18:34:31 on dist-test-slave-1xrh"
server_key: "f65322bb79d395b079b70a5d97cd4d1b"
server_key_iv: "a3cb595100895b0e72d16a217bd022b2"
server_key_version: "encryptionkey@0"
I20260806 18:34:31.246186 25407 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.003s
I20260806 18:34:31.248729 25422 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 18:34:31.249934 25407 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260806 18:34:31.250079 25407 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786041246094603-20232-0/minicluster-data/ts-2/data,/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786041246094603-20232-0/minicluster-data/ts-2/wal
uuid: "4850c1053d124036921de52053cfff53"
format_stamp: "Formatted at 2026-08-06 18:34:31 on dist-test-slave-1xrh"
server_key: "f65322bb79d395b079b70a5d97cd4d1b"
server_key_iv: "a3cb595100895b0e72d16a217bd022b2"
server_key_version: "encryptionkey@0"
I20260806 18:34:31.250188 25407 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786041246094603-20232-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786041246094603-20232-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786041246094603-20232-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 18:34:31.254451 25407 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 18:34:31.255014 25407 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 18:34:31.255227 25407 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 18:34:31.255731 25407 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 18:34:31.256573 25407 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 18:34:31.256640 25407 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 18:34:31.256705 25407 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 18:34:31.256758 25407 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 18:34:31.265843 25407 rpc_server.cc:307] RPC server started. Bound to: 127.19.194.3:41899
I20260806 18:34:31.265872 25538 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.194.3:41899 every 8 connection(s)
I20260806 18:34:31.266682 25407 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786041246094603-20232-0/minicluster-data/ts-2/data/info.pb
I20260806 18:34:31.269840 20232 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJW69tn/build/debug/bin/kudu as pid 25407
I20260806 18:34:31.269927 20232 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786041246094603-20232-0/minicluster-data/ts-2/wal/instance
I20260806 18:34:31.270185 20232 external_mini_cluster.cc:1428] Setting key dc79089153f9bf9a539d2077bde76731
I20260806 18:34:31.272886 25112 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:34:31.268401 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.194.3:53343 (local address 127.19.194.62:42911)
0806 18:34:31.268560 (+ 159us) server_negotiation.cc:206] Beginning negotiation
0806 18:34:31.268565 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 18:34:31.269608 (+ 1043us) server_negotiation.cc:407] Connection header received
0806 18:34:31.270957 (+ 1349us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:34:31.270961 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:34:31.271008 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:34:31.271114 (+ 106us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:34:31.272509 (+ 1395us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:34:31.272512 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:34:31.272514 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 18:34:31.272525 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:34:31.272545 (+ 20us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 18:34:31.272553 (+ 8us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:34:31.272622 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0806 18:34:31.272753 (+ 131us) server_negotiation.cc:299] Negotiation successful
0806 18:34:31.272788 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260806 18:34:31.273742 25541 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:34:31.268815 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.194.62:42911 (local address 127.19.194.3:53343)
0806 18:34:31.269470 (+ 655us) negotiation.cc:107] Waiting for socket to connect
0806 18:34:31.269508 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0806 18:34:31.270769 (+ 1261us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:34:31.271343 (+ 574us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:34:31.271354 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:34:31.271486 (+ 132us) client_negotiation.cc:189] Negotiated authn=SASL
0806 18:34:31.272190 (+ 704us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0806 18:34:31.272193 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0806 18:34:31.272245 (+ 52us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0806 18:34:31.272335 (+ 90us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 18:34:31.272630 (+ 295us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 18:34:31.272638 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 18:34:31.272642 (+ 4us) client_negotiation.cc:769] Sending connection context
0806 18:34:31.272769 (+ 127us) client_negotiation.cc:240] Negotiation successful
0806 18:34:31.272972 (+ 203us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":255,"thread_start_us":119,"threads_started":1}
I20260806 18:34:31.274868 25539 heartbeater.cc:347] Connected to a master server at 127.19.194.62:42911
I20260806 18:34:31.275089 25539 heartbeater.cc:464] Registering TS with master...
I20260806 18:34:31.275499 25539 heartbeater.cc:511] Master 127.19.194.62:42911 requested a full tablet report, sending...
I20260806 18:34:31.276427 25070 ts_manager.cc:194] Registered new tserver with Master: 4850c1053d124036921de52053cfff53 (127.19.194.3:41899)
I20260806 18:34:31.284143 20232 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260806 18:34:31.287408 25112 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:34:31.285933 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51892 (local address 127.19.194.62:42911)
0806 18:34:31.286075 (+ 142us) server_negotiation.cc:206] Beginning negotiation
0806 18:34:31.286079 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 18:34:31.286254 (+ 175us) server_negotiation.cc:407] Connection header received
0806 18:34:31.286377 (+ 123us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:34:31.286379 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:34:31.286410 (+ 31us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:34:31.286472 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:34:31.287024 (+ 552us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:34:31.287027 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:34:31.287030 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 18:34:31.287041 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:34:31.287059 (+ 18us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 18:34:31.287066 (+ 7us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:34:31.287169 (+ 103us) server_negotiation.cc:1035] Waiting for connection context
0806 18:34:31.287237 (+ 68us) server_negotiation.cc:299] Negotiation successful
0806 18:34:31.287268 (+ 31us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260806 18:34:31.290004 25070 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:51892:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260806 18:34:31.293045 25070 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 18:34:31.304302 25551 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:34:31.301105 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.194.3:41899 (local address 127.0.0.1:56450)
0806 18:34:31.301551 (+ 446us) negotiation.cc:107] Waiting for socket to connect
0806 18:34:31.301586 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0806 18:34:31.301660 (+ 74us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:34:31.302845 (+ 1185us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:34:31.302851 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:34:31.302918 (+ 67us) client_negotiation.cc:189] Negotiated authn=SASL
0806 18:34:31.303460 (+ 542us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0806 18:34:31.303465 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0806 18:34:31.303488 (+ 23us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0806 18:34:31.303571 (+ 83us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 18:34:31.303978 (+ 407us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 18:34:31.303989 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 18:34:31.304000 (+ 11us) client_negotiation.cc:769] Sending connection context
0806 18:34:31.304112 (+ 112us) client_negotiation.cc:240] Negotiation successful
0806 18:34:31.304176 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":289,"thread_start_us":71,"threads_started":1}
I20260806 18:34:31.304996 25554 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:34:31.301251 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.194.2:34431 (local address 127.0.0.1:59272)
0806 18:34:31.302132 (+ 881us) negotiation.cc:107] Waiting for socket to connect
0806 18:34:31.302148 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0806 18:34:31.302230 (+ 82us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:34:31.303043 (+ 813us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:34:31.303049 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:34:31.303070 (+ 21us) client_negotiation.cc:189] Negotiated authn=SASL
0806 18:34:31.303601 (+ 531us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0806 18:34:31.303603 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0806 18:34:31.303613 (+ 10us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0806 18:34:31.303638 (+ 25us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 18:34:31.304750 (+ 1112us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 18:34:31.304752 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 18:34:31.304755 (+ 3us) client_negotiation.cc:769] Sending connection context
0806 18:34:31.304843 (+ 88us) client_negotiation.cc:240] Negotiation successful
0806 18:34:31.304876 (+ 33us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":822,"thread_start_us":88,"threads_started":1}
I20260806 18:34:31.305150 25557 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:34:31.302057 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56450 (local address 127.19.194.3:41899)
0806 18:34:31.302482 (+ 425us) server_negotiation.cc:206] Beginning negotiation
0806 18:34:31.302497 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0806 18:34:31.302526 (+ 29us) server_negotiation.cc:407] Connection header received
0806 18:34:31.302565 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:34:31.302569 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:34:31.302702 (+ 133us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:34:31.302802 (+ 100us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:34:31.303742 (+ 940us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:34:31.303748 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:34:31.303759 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 18:34:31.303771 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:34:31.303833 (+ 62us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 18:34:31.303872 (+ 39us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:34:31.303988 (+ 116us) server_negotiation.cc:1035] Waiting for connection context
0806 18:34:31.304914 (+ 926us) server_negotiation.cc:299] Negotiation successful
0806 18:34:31.304989 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":317,"thread_start_us":130,"threads_started":1}
I20260806 18:34:31.305477 25555 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:34:31.301759 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59272 (local address 127.19.194.2:34431)
0806 18:34:31.302634 (+ 875us) server_negotiation.cc:206] Beginning negotiation
0806 18:34:31.302642 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 18:34:31.302660 (+ 18us) server_negotiation.cc:407] Connection header received
0806 18:34:31.302701 (+ 41us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:34:31.302705 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:34:31.302883 (+ 178us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:34:31.303022 (+ 139us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:34:31.304193 (+ 1171us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:34:31.304201 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:34:31.304214 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 18:34:31.304229 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:34:31.304560 (+ 331us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 18:34:31.304600 (+ 40us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:34:31.304703 (+ 103us) server_negotiation.cc:1035] Waiting for connection context
0806 18:34:31.305238 (+ 535us) server_negotiation.cc:299] Negotiation successful
0806 18:34:31.305322 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":790,"thread_start_us":107,"threads_started":1}
I20260806 18:34:31.305536 25556 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:34:31.301972 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47294 (local address 127.19.194.1:38513)
0806 18:34:31.302448 (+ 476us) server_negotiation.cc:206] Beginning negotiation
0806 18:34:31.302466 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0806 18:34:31.303734 (+ 1268us) server_negotiation.cc:407] Connection header received
0806 18:34:31.303804 (+ 70us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:34:31.303808 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:34:31.303952 (+ 144us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:34:31.304091 (+ 139us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:34:31.304781 (+ 690us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:34:31.304787 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:34:31.304798 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 18:34:31.304810 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:34:31.304848 (+ 38us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 18:34:31.304879 (+ 31us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:34:31.304993 (+ 114us) server_negotiation.cc:1035] Waiting for connection context
0806 18:34:31.305311 (+ 318us) server_negotiation.cc:299] Negotiation successful
0806 18:34:31.305385 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":373,"thread_start_us":161,"threads_started":1}
I20260806 18:34:31.305823 25553 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:34:31.301769 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.194.1:38513 (local address 127.0.0.1:47294)
0806 18:34:31.303497 (+ 1728us) negotiation.cc:107] Waiting for socket to connect
0806 18:34:31.303512 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0806 18:34:31.303617 (+ 105us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:34:31.304081 (+ 464us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:34:31.304084 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:34:31.304099 (+ 15us) client_negotiation.cc:189] Negotiated authn=SASL
0806 18:34:31.304548 (+ 449us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0806 18:34:31.304550 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0806 18:34:31.304561 (+ 11us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0806 18:34:31.304587 (+ 26us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 18:34:31.305038 (+ 451us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 18:34:31.305041 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 18:34:31.305045 (+ 4us) client_negotiation.cc:769] Sending connection context
0806 18:34:31.305678 (+ 633us) client_negotiation.cc:240] Negotiation successful
0806 18:34:31.305715 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1650,"thread_start_us":96,"threads_started":1}
I20260806 18:34:31.308149 25470 tablet_service.cc:1467] Processing CreateTablet for tablet 3bbcb0a9afc349e0bd4b9b90bff1b4c9 (DEFAULT_TABLE table=test-table [id=c845b963ab1847c3b0c7a6a38d5d2d60]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 18:34:31.308213 25334 tablet_service.cc:1467] Processing CreateTablet for tablet 3bbcb0a9afc349e0bd4b9b90bff1b4c9 (DEFAULT_TABLE table=test-table [id=c845b963ab1847c3b0c7a6a38d5d2d60]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 18:34:31.308208 25197 tablet_service.cc:1467] Processing CreateTablet for tablet 3bbcb0a9afc349e0bd4b9b90bff1b4c9 (DEFAULT_TABLE table=test-table [id=c845b963ab1847c3b0c7a6a38d5d2d60]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 18:34:31.309146 25470 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3bbcb0a9afc349e0bd4b9b90bff1b4c9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 18:34:31.309219 25334 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3bbcb0a9afc349e0bd4b9b90bff1b4c9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 18:34:31.309219 25197 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3bbcb0a9afc349e0bd4b9b90bff1b4c9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 18:34:31.317420 25558 tablet_bootstrap.cc:492] T 3bbcb0a9afc349e0bd4b9b90bff1b4c9 P 24f7daf40ef341f9b96a4c8ddcbe9fde: Bootstrap starting.
I20260806 18:34:31.318022 25560 tablet_bootstrap.cc:492] T 3bbcb0a9afc349e0bd4b9b90bff1b4c9 P 3557d45d2daa4d65b3cfe14511b486e6: Bootstrap starting.
I20260806 18:34:31.318140 25559 tablet_bootstrap.cc:492] T 3bbcb0a9afc349e0bd4b9b90bff1b4c9 P 4850c1053d124036921de52053cfff53: Bootstrap starting.
I20260806 18:34:31.319868 25558 tablet_bootstrap.cc:654] T 3bbcb0a9afc349e0bd4b9b90bff1b4c9 P 24f7daf40ef341f9b96a4c8ddcbe9fde: Neither blocks nor log segments found. Creating new log.
I20260806 18:34:31.320061 25559 tablet_bootstrap.cc:654] T 3bbcb0a9afc349e0bd4b9b90bff1b4c9 P 4850c1053d124036921de52053cfff53: Neither blocks nor log segments found. Creating new log.
I20260806 18:34:31.320165 25560 tablet_bootstrap.cc:654] T 3bbcb0a9afc349e0bd4b9b90bff1b4c9 P 3557d45d2daa4d65b3cfe14511b486e6: Neither blocks nor log segments found. Creating new log.
I20260806 18:34:31.320813 25558 log.cc:824] T 3bbcb0a9afc349e0bd4b9b90bff1b4c9 P 24f7daf40ef341f9b96a4c8ddcbe9fde: Log is configured to *not* fsync() on all Append() calls
I20260806 18:34:31.320813 25559 log.cc:824] T 3bbcb0a9afc349e0bd4b9b90bff1b4c9 P 4850c1053d124036921de52053cfff53: Log is configured to *not* fsync() on all Append() calls
I20260806 18:34:31.320859 25560 log.cc:824] T 3bbcb0a9afc349e0bd4b9b90bff1b4c9 P 3557d45d2daa4d65b3cfe14511b486e6: Log is configured to *not* fsync() on all Append() calls
I20260806 18:34:31.322681 25560 tablet_bootstrap.cc:492] T 3bbcb0a9afc349e0bd4b9b90bff1b4c9 P 3557d45d2daa4d65b3cfe14511b486e6: No bootstrap required, opened a new log
I20260806 18:34:31.322803 25558 tablet_bootstrap.cc:492] T 3bbcb0a9afc349e0bd4b9b90bff1b4c9 P 24f7daf40ef341f9b96a4c8ddcbe9fde: No bootstrap required, opened a new log
I20260806 18:34:31.322927 25560 ts_tablet_manager.cc:1397] T 3bbcb0a9afc349e0bd4b9b90bff1b4c9 P 3557d45d2daa4d65b3cfe14511b486e6: Time spent bootstrapping tablet: real 0.005s user 0.003s sys 0.000s
I20260806 18:34:31.323006 25558 ts_tablet_manager.cc:1397] T 3bbcb0a9afc349e0bd4b9b90bff1b4c9 P 24f7daf40ef341f9b96a4c8ddcbe9fde: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260806 18:34:31.323058 25559 tablet_bootstrap.cc:492] T 3bbcb0a9afc349e0bd4b9b90bff1b4c9 P 4850c1053d124036921de52053cfff53: No bootstrap required, opened a new log
I20260806 18:34:31.323287 25559 ts_tablet_manager.cc:1397] T 3bbcb0a9afc349e0bd4b9b90bff1b4c9 P 4850c1053d124036921de52053cfff53: Time spent bootstrapping tablet: real 0.005s user 0.001s sys 0.003s
I20260806 18:34:31.326857 25560 raft_consensus.cc:348] T 3bbcb0a9afc349e0bd4b9b90bff1b4c9 P 3557d45d2daa4d65b3cfe14511b486e6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4850c1053d124036921de52053cfff53" member_type: VOTER last_known_addr { host: "127.19.194.3" port: 41899 } } peers { permanent_uuid: "3557d45d2daa4d65b3cfe14511b486e6" member_type: VOTER last_known_addr { host: "127.19.194.2" port: 34431 } } peers { permanent_uuid: "24f7daf40ef341f9b96a4c8ddcbe9fde" member_type: VOTER last_known_addr { host: "127.19.194.1" port: 38513 } }
I20260806 18:34:31.327211 25560 raft_consensus.cc:374] T 3bbcb0a9afc349e0bd4b9b90bff1b4c9 P 3557d45d2daa4d65b3cfe14511b486e6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 18:34:31.327283 25560 raft_consensus.cc:729] T 3bbcb0a9afc349e0bd4b9b90bff1b4c9 P 3557d45d2daa4d65b3cfe14511b486e6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3557d45d2daa4d65b3cfe14511b486e6, State: Initialized, Role: FOLLOWER
I20260806 18:34:31.327621 25558 raft_consensus.cc:348] T 3bbcb0a9afc349e0bd4b9b90bff1b4c9 P 24f7daf40ef341f9b96a4c8ddcbe9fde [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4850c1053d124036921de52053cfff53" member_type: VOTER last_known_addr { host: "127.19.194.3" port: 41899 } } peers { permanent_uuid: "3557d45d2daa4d65b3cfe14511b486e6" member_type: VOTER last_known_addr { host: "127.19.194.2" port: 34431 } } peers { permanent_uuid: "24f7daf40ef341f9b96a4c8ddcbe9fde" member_type: VOTER last_known_addr { host: "127.19.194.1" port: 38513 } }
I20260806 18:34:31.327621 25559 raft_consensus.cc:348] T 3bbcb0a9afc349e0bd4b9b90bff1b4c9 P 4850c1053d124036921de52053cfff53 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4850c1053d124036921de52053cfff53" member_type: VOTER last_known_addr { host: "127.19.194.3" port: 41899 } } peers { permanent_uuid: "3557d45d2daa4d65b3cfe14511b486e6" member_type: VOTER last_known_addr { host: "127.19.194.2" port: 34431 } } peers { permanent_uuid: "24f7daf40ef341f9b96a4c8ddcbe9fde" member_type: VOTER last_known_addr { host: "127.19.194.1" port: 38513 } }
I20260806 18:34:31.327745 25560 consensus_queue.cc:260] T 3bbcb0a9afc349e0bd4b9b90bff1b4c9 P 3557d45d2daa4d65b3cfe14511b486e6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4850c1053d124036921de52053cfff53" member_type: VOTER last_known_addr { host: "127.19.194.3" port: 41899 } } peers { permanent_uuid: "3557d45d2daa4d65b3cfe14511b486e6" member_type: VOTER last_known_addr { host: "127.19.194.2" port: 34431 } } peers { permanent_uuid: "24f7daf40ef341f9b96a4c8ddcbe9fde" member_type: VOTER last_known_addr { host: "127.19.194.1" port: 38513 } }
I20260806 18:34:31.327862 25558 raft_consensus.cc:374] T 3bbcb0a9afc349e0bd4b9b90bff1b4c9 P 24f7daf40ef341f9b96a4c8ddcbe9fde [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 18:34:31.327862 25559 raft_consensus.cc:374] T 3bbcb0a9afc349e0bd4b9b90bff1b4c9 P 4850c1053d124036921de52053cfff53 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 18:34:31.327920 25558 raft_consensus.cc:729] T 3bbcb0a9afc349e0bd4b9b90bff1b4c9 P 24f7daf40ef341f9b96a4c8ddcbe9fde [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 24f7daf40ef341f9b96a4c8ddcbe9fde, State: Initialized, Role: FOLLOWER
I20260806 18:34:31.327920 25559 raft_consensus.cc:729] T 3bbcb0a9afc349e0bd4b9b90bff1b4c9 P 4850c1053d124036921de52053cfff53 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4850c1053d124036921de52053cfff53, State: Initialized, Role: FOLLOWER
I20260806 18:34:31.328320 25559 consensus_queue.cc:260] T 3bbcb0a9afc349e0bd4b9b90bff1b4c9 P 4850c1053d124036921de52053cfff53 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4850c1053d124036921de52053cfff53" member_type: VOTER last_known_addr { host: "127.19.194.3" port: 41899 } } peers { permanent_uuid: "3557d45d2daa4d65b3cfe14511b486e6" member_type: VOTER last_known_addr { host: "127.19.194.2" port: 34431 } } peers { permanent_uuid: "24f7daf40ef341f9b96a4c8ddcbe9fde" member_type: VOTER last_known_addr { host: "127.19.194.1" port: 38513 } }
I20260806 18:34:31.328320 25558 consensus_queue.cc:260] T 3bbcb0a9afc349e0bd4b9b90bff1b4c9 P 24f7daf40ef341f9b96a4c8ddcbe9fde [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4850c1053d124036921de52053cfff53" member_type: VOTER last_known_addr { host: "127.19.194.3" port: 41899 } } peers { permanent_uuid: "3557d45d2daa4d65b3cfe14511b486e6" member_type: VOTER last_known_addr { host: "127.19.194.2" port: 34431 } } peers { permanent_uuid: "24f7daf40ef341f9b96a4c8ddcbe9fde" member_type: VOTER last_known_addr { host: "127.19.194.1" port: 38513 } }
I20260806 18:34:31.328749 25560 ts_tablet_manager.cc:1428] T 3bbcb0a9afc349e0bd4b9b90bff1b4c9 P 3557d45d2daa4d65b3cfe14511b486e6: Time spent starting tablet: real 0.006s user 0.006s sys 0.000s
I20260806 18:34:31.329082 25404 heartbeater.cc:503] Master 127.19.194.62:42911 was elected leader, sending a full tablet report...
I20260806 18:34:31.329226 25559 ts_tablet_manager.cc:1428] T 3bbcb0a9afc349e0bd4b9b90bff1b4c9 P 4850c1053d124036921de52053cfff53: Time spent starting tablet: real 0.006s user 0.005s sys 0.000s
I20260806 18:34:31.329239 25539 heartbeater.cc:503] Master 127.19.194.62:42911 was elected leader, sending a full tablet report...
I20260806 18:34:31.329366 25558 ts_tablet_manager.cc:1428] T 3bbcb0a9afc349e0bd4b9b90bff1b4c9 P 24f7daf40ef341f9b96a4c8ddcbe9fde: Time spent starting tablet: real 0.006s user 0.008s sys 0.000s
I20260806 18:34:31.330116 25267 heartbeater.cc:503] Master 127.19.194.62:42911 was elected leader, sending a full tablet report...
W20260806 18:34:31.347812 25405 tablet.cc:2369] T 3bbcb0a9afc349e0bd4b9b90bff1b4c9 P 3557d45d2daa4d65b3cfe14511b486e6: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 18:34:31.367875 25566 raft_consensus.cc:482] T 3bbcb0a9afc349e0bd4b9b90bff1b4c9 P 4850c1053d124036921de52053cfff53 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 18:34:31.368063 25566 raft_consensus.cc:504] T 3bbcb0a9afc349e0bd4b9b90bff1b4c9 P 4850c1053d124036921de52053cfff53 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4850c1053d124036921de52053cfff53" member_type: VOTER last_known_addr { host: "127.19.194.3" port: 41899 } } peers { permanent_uuid: "3557d45d2daa4d65b3cfe14511b486e6" member_type: VOTER last_known_addr { host: "127.19.194.2" port: 34431 } } peers { permanent_uuid: "24f7daf40ef341f9b96a4c8ddcbe9fde" member_type: VOTER last_known_addr { host: "127.19.194.1" port: 38513 } }
I20260806 18:34:31.369136 25566 leader_election.cc:288] T 3bbcb0a9afc349e0bd4b9b90bff1b4c9 P 4850c1053d124036921de52053cfff53 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3557d45d2daa4d65b3cfe14511b486e6 (127.19.194.2:34431), 24f7daf40ef341f9b96a4c8ddcbe9fde (127.19.194.1:38513)
I20260806 18:34:31.371069 25555 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:34:31.369468 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.194.3:45261 (local address 127.19.194.2:34431)
0806 18:34:31.369629 (+ 161us) server_negotiation.cc:206] Beginning negotiation
0806 18:34:31.369635 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 18:34:31.369651 (+ 16us) server_negotiation.cc:407] Connection header received
0806 18:34:31.369701 (+ 50us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:34:31.369704 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:34:31.369744 (+ 40us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:34:31.369853 (+ 109us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:34:31.370676 (+ 823us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:34:31.370680 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:34:31.370682 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 18:34:31.370697 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:34:31.370720 (+ 23us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 18:34:31.370728 (+ 8us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:34:31.370824 (+ 96us) server_negotiation.cc:1035] Waiting for connection context
0806 18:34:31.370902 (+ 78us) server_negotiation.cc:299] Negotiation successful
0806 18:34:31.370943 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260806 18:34:31.371069 25541 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:34:31.369345 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.194.2:34431 (local address 127.19.194.3:45261)
0806 18:34:31.369483 (+ 138us) negotiation.cc:107] Waiting for socket to connect
0806 18:34:31.369502 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0806 18:34:31.369606 (+ 104us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:34:31.369881 (+ 275us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:34:31.369884 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:34:31.369901 (+ 17us) client_negotiation.cc:189] Negotiated authn=SASL
0806 18:34:31.370509 (+ 608us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0806 18:34:31.370510 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0806 18:34:31.370520 (+ 10us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0806 18:34:31.370544 (+ 24us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 18:34:31.370819 (+ 275us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 18:34:31.370822 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 18:34:31.370826 (+ 4us) client_negotiation.cc:769] Sending connection context
0806 18:34:31.370903 (+ 77us) client_negotiation.cc:240] Negotiation successful
0806 18:34:31.370941 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":53}
I20260806 18:34:31.371580 25556 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:34:31.369815 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.194.3:56413 (local address 127.19.194.1:38513)
0806 18:34:31.370029 (+ 214us) server_negotiation.cc:206] Beginning negotiation
0806 18:34:31.370036 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 18:34:31.370166 (+ 130us) server_negotiation.cc:407] Connection header received
0806 18:34:31.370245 (+ 79us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:34:31.370247 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:34:31.370279 (+ 32us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:34:31.370380 (+ 101us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:34:31.371054 (+ 674us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:34:31.371058 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:34:31.371060 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 18:34:31.371073 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:34:31.371099 (+ 26us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 18:34:31.371108 (+ 9us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:34:31.371249 (+ 141us) server_negotiation.cc:1035] Waiting for connection context
0806 18:34:31.371400 (+ 151us) server_negotiation.cc:299] Negotiation successful
0806 18:34:31.371444 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":105}
I20260806 18:34:31.371809 25354 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3bbcb0a9afc349e0bd4b9b90bff1b4c9" candidate_uuid: "4850c1053d124036921de52053cfff53" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3557d45d2daa4d65b3cfe14511b486e6" is_pre_election: true
I20260806 18:34:31.371920 25567 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:34:31.369661 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.194.1:38513 (local address 127.19.194.3:56413)
0806 18:34:31.370041 (+ 380us) negotiation.cc:107] Waiting for socket to connect
0806 18:34:31.370052 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0806 18:34:31.370169 (+ 117us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:34:31.370421 (+ 252us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:34:31.370426 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:34:31.370444 (+ 18us) client_negotiation.cc:189] Negotiated authn=SASL
0806 18:34:31.370882 (+ 438us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0806 18:34:31.370884 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0806 18:34:31.370894 (+ 10us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0806 18:34:31.370919 (+ 25us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 18:34:31.371301 (+ 382us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 18:34:31.371305 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 18:34:31.371308 (+ 3us) client_negotiation.cc:769] Sending connection context
0806 18:34:31.371740 (+ 432us) client_negotiation.cc:240] Negotiation successful
0806 18:34:31.371780 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":319,"thread_start_us":138,"threads_started":1}
I20260806 18:34:31.372129 25354 raft_consensus.cc:2454] T 3bbcb0a9afc349e0bd4b9b90bff1b4c9 P 3557d45d2daa4d65b3cfe14511b486e6 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4850c1053d124036921de52053cfff53 in term 0.
I20260806 18:34:31.372545 25220 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3bbcb0a9afc349e0bd4b9b90bff1b4c9" candidate_uuid: "4850c1053d124036921de52053cfff53" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "24f7daf40ef341f9b96a4c8ddcbe9fde" is_pre_election: true
I20260806 18:34:31.372586 25424 leader_election.cc:302] T 3bbcb0a9afc349e0bd4b9b90bff1b4c9 P 4850c1053d124036921de52053cfff53 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3557d45d2daa4d65b3cfe14511b486e6, 4850c1053d124036921de52053cfff53; no voters:
I20260806 18:34:31.372826 25220 raft_consensus.cc:2454] T 3bbcb0a9afc349e0bd4b9b90bff1b4c9 P 24f7daf40ef341f9b96a4c8ddcbe9fde [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4850c1053d124036921de52053cfff53 in term 0.
I20260806 18:34:31.372880 25566 raft_consensus.cc:2781] T 3bbcb0a9afc349e0bd4b9b90bff1b4c9 P 4850c1053d124036921de52053cfff53 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 18:34:31.372978 25566 raft_consensus.cc:482] T 3bbcb0a9afc349e0bd4b9b90bff1b4c9 P 4850c1053d124036921de52053cfff53 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 18:34:31.373027 25566 raft_consensus.cc:3037] T 3bbcb0a9afc349e0bd4b9b90bff1b4c9 P 4850c1053d124036921de52053cfff53 [term 0 FOLLOWER]: Advancing to term 1
I20260806 18:34:31.374145 25566 raft_consensus.cc:504] T 3bbcb0a9afc349e0bd4b9b90bff1b4c9 P 4850c1053d124036921de52053cfff53 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4850c1053d124036921de52053cfff53" member_type: VOTER last_known_addr { host: "127.19.194.3" port: 41899 } } peers { permanent_uuid: "3557d45d2daa4d65b3cfe14511b486e6" member_type: VOTER last_known_addr { host: "127.19.194.2" port: 34431 } } peers { permanent_uuid: "24f7daf40ef341f9b96a4c8ddcbe9fde" member_type: VOTER last_known_addr { host: "127.19.194.1" port: 38513 } }
I20260806 18:34:31.374507 25566 leader_election.cc:288] T 3bbcb0a9afc349e0bd4b9b90bff1b4c9 P 4850c1053d124036921de52053cfff53 [CANDIDATE]: Term 1 election: Requested vote from peers 3557d45d2daa4d65b3cfe14511b486e6 (127.19.194.2:34431), 24f7daf40ef341f9b96a4c8ddcbe9fde (127.19.194.1:38513)
I20260806 18:34:31.374863 25354 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3bbcb0a9afc349e0bd4b9b90bff1b4c9" candidate_uuid: "4850c1053d124036921de52053cfff53" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3557d45d2daa4d65b3cfe14511b486e6"
I20260806 18:34:31.374882 25220 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3bbcb0a9afc349e0bd4b9b90bff1b4c9" candidate_uuid: "4850c1053d124036921de52053cfff53" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "24f7daf40ef341f9b96a4c8ddcbe9fde"
I20260806 18:34:31.374989 25220 raft_consensus.cc:3037] T 3bbcb0a9afc349e0bd4b9b90bff1b4c9 P 24f7daf40ef341f9b96a4c8ddcbe9fde [term 0 FOLLOWER]: Advancing to term 1
I20260806 18:34:31.374989 25354 raft_consensus.cc:3037] T 3bbcb0a9afc349e0bd4b9b90bff1b4c9 P 3557d45d2daa4d65b3cfe14511b486e6 [term 0 FOLLOWER]: Advancing to term 1
I20260806 18:34:31.376291 25220 raft_consensus.cc:2454] T 3bbcb0a9afc349e0bd4b9b90bff1b4c9 P 24f7daf40ef341f9b96a4c8ddcbe9fde [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4850c1053d124036921de52053cfff53 in term 1.
I20260806 18:34:31.376291 25354 raft_consensus.cc:2454] T 3bbcb0a9afc349e0bd4b9b90bff1b4c9 P 3557d45d2daa4d65b3cfe14511b486e6 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4850c1053d124036921de52053cfff53 in term 1.
I20260806 18:34:31.376575 25424 leader_election.cc:302] T 3bbcb0a9afc349e0bd4b9b90bff1b4c9 P 4850c1053d124036921de52053cfff53 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 24f7daf40ef341f9b96a4c8ddcbe9fde, 4850c1053d124036921de52053cfff53; no voters:
I20260806 18:34:31.376766 25566 raft_consensus.cc:2781] T 3bbcb0a9afc349e0bd4b9b90bff1b4c9 P 4850c1053d124036921de52053cfff53 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 18:34:31.376994 25566 raft_consensus.cc:686] T 3bbcb0a9afc349e0bd4b9b90bff1b4c9 P 4850c1053d124036921de52053cfff53 [term 1 LEADER]: Becoming Leader. State: Replica: 4850c1053d124036921de52053cfff53, State: Running, Role: LEADER
I20260806 18:34:31.377318 25566 consensus_queue.cc:237] T 3bbcb0a9afc349e0bd4b9b90bff1b4c9 P 4850c1053d124036921de52053cfff53 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4850c1053d124036921de52053cfff53" member_type: VOTER last_known_addr { host: "127.19.194.3" port: 41899 } } peers { permanent_uuid: "3557d45d2daa4d65b3cfe14511b486e6" member_type: VOTER last_known_addr { host: "127.19.194.2" port: 34431 } } peers { permanent_uuid: "24f7daf40ef341f9b96a4c8ddcbe9fde" member_type: VOTER last_known_addr { host: "127.19.194.1" port: 38513 } }
I20260806 18:34:31.380620 25069 catalog_manager.cc:5495] T 3bbcb0a9afc349e0bd4b9b90bff1b4c9 P 4850c1053d124036921de52053cfff53 reported cstate change: term changed from 0 to 1, leader changed from <none> to 4850c1053d124036921de52053cfff53 (127.19.194.3). New cstate: current_term: 1 leader_uuid: "4850c1053d124036921de52053cfff53" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4850c1053d124036921de52053cfff53" member_type: VOTER last_known_addr { host: "127.19.194.3" port: 41899 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3557d45d2daa4d65b3cfe14511b486e6" member_type: VOTER last_known_addr { host: "127.19.194.2" port: 34431 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "24f7daf40ef341f9b96a4c8ddcbe9fde" member_type: VOTER last_known_addr { host: "127.19.194.1" port: 38513 } health_report { overall_health: UNKNOWN } } }
I20260806 18:34:31.404655 25557 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:34:31.403020 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56452 (local address 127.19.194.3:41899)
0806 18:34:31.403205 (+ 185us) server_negotiation.cc:206] Beginning negotiation
0806 18:34:31.403212 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 18:34:31.403228 (+ 16us) server_negotiation.cc:407] Connection header received
0806 18:34:31.403414 (+ 186us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:34:31.403416 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:34:31.403483 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:34:31.403575 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:34:31.404218 (+ 643us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:34:31.404222 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:34:31.404225 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 18:34:31.404236 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:34:31.404254 (+ 18us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 18:34:31.404270 (+ 16us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:34:31.404363 (+ 93us) server_negotiation.cc:1035] Waiting for connection context
0806 18:34:31.404477 (+ 114us) server_negotiation.cc:299] Negotiation successful
0806 18:34:31.404518 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260806 18:34:31.410351 25220 raft_consensus.cc:1261] T 3bbcb0a9afc349e0bd4b9b90bff1b4c9 P 24f7daf40ef341f9b96a4c8ddcbe9fde [term 1 FOLLOWER]: Refusing update from remote peer 4850c1053d124036921de52053cfff53: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 18:34:31.410351 25354 raft_consensus.cc:1261] T 3bbcb0a9afc349e0bd4b9b90bff1b4c9 P 3557d45d2daa4d65b3cfe14511b486e6 [term 1 FOLLOWER]: Refusing update from remote peer 4850c1053d124036921de52053cfff53: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 18:34:31.411051 25566 consensus_queue.cc:1035] T 3bbcb0a9afc349e0bd4b9b90bff1b4c9 P 4850c1053d124036921de52053cfff53 [LEADER]: Connected to new peer: Peer: permanent_uuid: "24f7daf40ef341f9b96a4c8ddcbe9fde" member_type: VOTER last_known_addr { host: "127.19.194.1" port: 38513 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 18:34:31.411288 25568 consensus_queue.cc:1035] T 3bbcb0a9afc349e0bd4b9b90bff1b4c9 P 4850c1053d124036921de52053cfff53 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3557d45d2daa4d65b3cfe14511b486e6" member_type: VOTER last_known_addr { host: "127.19.194.2" port: 34431 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 18:34:31.419127 25572 mvcc.cc:205] Tried to move back new op lower bound from 7315625047689715712 to 7315625047564320768. Current Snapshot: MvccSnapshot[applied={T|T < 7315625047689715712}]
I20260806 18:34:31.421495 25573 mvcc.cc:205] Tried to move back new op lower bound from 7315625047689715712 to 7315625047564320768. Current Snapshot: MvccSnapshot[applied={T|T < 7315625047689715712}]
I20260806 18:34:31.423228 25571 mvcc.cc:205] Tried to move back new op lower bound from 7315625047689715712 to 7315625047564320768. Current Snapshot: MvccSnapshot[applied={T|T < 7315625047689715712 or (T in {7315625047689715712})}]
I20260806 18:34:31.426880 25069 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:51892:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 18:34:31.427008 25069 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:51892:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 18:34:31.429325 25069 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 901fc9604d1d4e94928432c374cf7f44: Sending DeleteTablet for 3 replicas of tablet 3bbcb0a9afc349e0bd4b9b90bff1b4c9
I20260806 18:34:31.430085 25470 tablet_service.cc:1514] Processing DeleteTablet for tablet 3bbcb0a9afc349e0bd4b9b90bff1b4c9 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 18:34:31 UTC) from {username='slave'} at 127.0.0.1:56450
I20260806 18:34:31.430154 25334 tablet_service.cc:1514] Processing DeleteTablet for tablet 3bbcb0a9afc349e0bd4b9b90bff1b4c9 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 18:34:31 UTC) from {username='slave'} at 127.0.0.1:59272
I20260806 18:34:31.430549 25580 tablet_replica.cc:330] T 3bbcb0a9afc349e0bd4b9b90bff1b4c9 P 4850c1053d124036921de52053cfff53: stopping tablet replica
I20260806 18:34:31.430555 25197 tablet_service.cc:1514] Processing DeleteTablet for tablet 3bbcb0a9afc349e0bd4b9b90bff1b4c9 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 18:34:31 UTC) from {username='slave'} at 127.0.0.1:47294
I20260806 18:34:31.430826 25581 tablet_replica.cc:330] T 3bbcb0a9afc349e0bd4b9b90bff1b4c9 P 3557d45d2daa4d65b3cfe14511b486e6: stopping tablet replica
I20260806 18:34:31.430948 25582 tablet_replica.cc:330] T 3bbcb0a9afc349e0bd4b9b90bff1b4c9 P 24f7daf40ef341f9b96a4c8ddcbe9fde: stopping tablet replica
I20260806 18:34:31.431270 25582 raft_consensus.cc:2229] T 3bbcb0a9afc349e0bd4b9b90bff1b4c9 P 24f7daf40ef341f9b96a4c8ddcbe9fde [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 18:34:31.431354 20232 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJW69tn/build/debug/bin/kudu with pid 25133
I20260806 18:34:31.431798 25580 raft_consensus.cc:2229] T 3bbcb0a9afc349e0bd4b9b90bff1b4c9 P 4850c1053d124036921de52053cfff53 [term 1 LEADER]: Raft consensus shutting down.
I20260806 18:34:31.432029 25581 raft_consensus.cc:2229] T 3bbcb0a9afc349e0bd4b9b90bff1b4c9 P 3557d45d2daa4d65b3cfe14511b486e6 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 18:34:31.432323 25581 raft_consensus.cc:2258] T 3bbcb0a9afc349e0bd4b9b90bff1b4c9 P 3557d45d2daa4d65b3cfe14511b486e6 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 18:34:31.432691 25580 raft_consensus.cc:2258] T 3bbcb0a9afc349e0bd4b9b90bff1b4c9 P 4850c1053d124036921de52053cfff53 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 18:34:31.434233 25580 ts_tablet_manager.cc:1905] T 3bbcb0a9afc349e0bd4b9b90bff1b4c9 P 4850c1053d124036921de52053cfff53: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 18:34:31.437075 25580 ts_tablet_manager.cc:1918] T 3bbcb0a9afc349e0bd4b9b90bff1b4c9 P 4850c1053d124036921de52053cfff53: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 18:34:31.437172 25580 log.cc:1196] T 3bbcb0a9afc349e0bd4b9b90bff1b4c9 P 4850c1053d124036921de52053cfff53: Deleting WAL directory at /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786041246094603-20232-0/minicluster-data/ts-2/wal/wals/3bbcb0a9afc349e0bd4b9b90bff1b4c9
I20260806 18:34:31.437506 25580 ts_tablet_manager.cc:1939] T 3bbcb0a9afc349e0bd4b9b90bff1b4c9 P 4850c1053d124036921de52053cfff53: Deleting consensus metadata
I20260806 18:34:31.438580 25581 ts_tablet_manager.cc:1905] T 3bbcb0a9afc349e0bd4b9b90bff1b4c9 P 3557d45d2daa4d65b3cfe14511b486e6: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 18:34:31.438599 25054 catalog_manager.cc:4841] TS 4850c1053d124036921de52053cfff53 (127.19.194.3:41899): tablet 3bbcb0a9afc349e0bd4b9b90bff1b4c9 (table test-table [id=c845b963ab1847c3b0c7a6a38d5d2d60]) successfully deleted
W20260806 18:34:31.440352 25055 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.19.194.1:38513 (error 108)
I20260806 18:34:31.441376 25553 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:34:31.441084 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.194.1:38513 (local address 127.0.0.1:47304)
0806 18:34:31.441195 (+ 111us) negotiation.cc:107] Waiting for socket to connect
0806 18:34:31.441273 (+ 78us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.194.1:38513: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":32}
I20260806 18:34:31.441434 20232 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJW69tn/build/debug/bin/kudu with pid 25270
W20260806 18:34:31.447340 25055 catalog_manager.cc:4584] TS 24f7daf40ef341f9b96a4c8ddcbe9fde (127.19.194.1:38513): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 3bbcb0a9afc349e0bd4b9b90bff1b4c9: Network error: Client connection negotiation failed: client connection to 127.19.194.1:38513: connect: Connection refused (error 111)
I20260806 18:34:31.448596 20232 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJW69tn/build/debug/bin/kudu with pid 25407
I20260806 18:34:31.449651 25553 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:34:31.448744 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.194.2:34431 (local address 127.0.0.1:59274)
0806 18:34:31.449444 (+ 700us) negotiation.cc:107] Waiting for socket to connect
0806 18:34:31.449528 (+ 84us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.194.2:34431: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":592}
I20260806 18:34:31.456221 20232 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJW69tn/build/debug/bin/kudu with pid 25039
2026-08-06T18:34:31Z chronyd exiting
[ OK ] SecurityITest.TestDisableAuthenticationEncryption (970 ms)
[ RUN ] SecurityITest.TestJwtMiniCluster
I20260806 18:34:31.478564 20232 mini_oidc.cc:152] Starting JWKS server
I20260806 18:34:31.480214 20232 webserver.cc:460] Webserver started at https://127.0.0.1:36213/ using document root <none> and password file <none>
I20260806 18:34:31.480345 20232 mini_oidc.cc:189] Starting OIDC Discovery server
I20260806 18:34:31.480566 20232 webserver.cc:460] Webserver started at http://127.0.0.1:41707/ using document root <none> and password file <none>
2026-08-06T18:34:31Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T18:34:31Z Disabled control of system clock
I20260806 18:34:31.489545 20232 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJW69tn/build/debug/bin/kudu
/tmp/dist-test-taskJW69tn/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786041246094603-20232-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786041246094603-20232-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786041246094603-20232-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786041246094603-20232-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.19.194.62:40233
--webserver_interface=127.19.194.62
--webserver_port=0
--builtin_ntp_servers=127.19.194.20:39085
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.19.194.62:40233
--enable_jwt_token_auth=true
--jwks_url=https://localhost:36213/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786041246094603-20232-0/testchainca.cert with env {}
W20260806 18:34:31.599272 25589 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 18:34:31.599546 25589 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 18:34:31.599586 25589 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 18:34:31.603440 25589 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 18:34:31.603511 25589 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 18:34:31.603528 25589 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 18:34:31.603543 25589 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 18:34:31.603559 25589 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260806 18:34:31.603574 25589 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260806 18:34:31.603586 25589 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:36213/jwks.json
I20260806 18:34:31.608929 25589 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.194.20:39085
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786041246094603-20232-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786041246094603-20232-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.19.194.62:40233
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.19.194.62:40233
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:36213/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786041246094603-20232-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.19.194.62
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786041246094603-20232-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25589
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786041246094603-20232-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 17:59:13 UTC on ccycloud.quasar-wvsnzx.root.comops.site
build id 4571028
I20260806 18:34:31.609848 25589 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 18:34:31.610749 25589 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 18:34:31.618830 25595 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:34:31.618844 25594 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:34:31.618834 25597 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 18:34:31.619222 25589 server_base.cc:1034] running on GCE node
I20260806 18:34:31.619761 25589 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 18:34:31.620781 25589 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 18:34:31.621968 25589 hybrid_clock.cc:648] HybridClock initialized: now 1786041271621957 us; error 38 us; skew 500 ppm
I20260806 18:34:31.623849 25589 webserver.cc:460] Webserver started at http://127.19.194.62:34777/ using document root <none> and password file <none>
I20260806 18:34:31.624415 25589 fs_manager.cc:359] Metadata directory not provided
I20260806 18:34:31.624513 25589 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 18:34:31.624742 25589 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 18:34:31.626694 25589 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786041246094603-20232-0/minicluster-data/master-0/data/instance:
uuid: "84961f79d4ef4ab1afb11a93d69ffae5"
format_stamp: "Formatted at 2026-08-06 18:34:31 on dist-test-slave-1xrh"
server_key: "00eea0a51010b12f8faa74e00c755ee8"
server_key_iv: "164d8153ba85a64b990dc291800a566a"
server_key_version: "encryptionkey@0"
I20260806 18:34:31.627326 25589 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786041246094603-20232-0/minicluster-data/master-0/wal/instance:
uuid: "84961f79d4ef4ab1afb11a93d69ffae5"
format_stamp: "Formatted at 2026-08-06 18:34:31 on dist-test-slave-1xrh"
server_key: "00eea0a51010b12f8faa74e00c755ee8"
server_key_iv: "164d8153ba85a64b990dc291800a566a"
server_key_version: "encryptionkey@0"
I20260806 18:34:31.630887 25589 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.000s
I20260806 18:34:31.633174 25603 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 18:34:31.634277 25589 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260806 18:34:31.634418 25589 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786041246094603-20232-0/minicluster-data/master-0/data,/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786041246094603-20232-0/minicluster-data/master-0/wal
uuid: "84961f79d4ef4ab1afb11a93d69ffae5"
format_stamp: "Formatted at 2026-08-06 18:34:31 on dist-test-slave-1xrh"
server_key: "00eea0a51010b12f8faa74e00c755ee8"
server_key_iv: "164d8153ba85a64b990dc291800a566a"
server_key_version: "encryptionkey@0"
I20260806 18:34:31.634523 25589 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786041246094603-20232-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786041246094603-20232-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786041246094603-20232-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 18:34:31.656961 25589 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 18:34:31.657620 25589 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 18:34:31.657816 25589 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 18:34:31.664601 25589 rpc_server.cc:307] RPC server started. Bound to: 127.19.194.62:40233
I20260806 18:34:31.664673 25655 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.194.62:40233 every 8 connection(s)
I20260806 18:34:31.665551 25589 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786041246094603-20232-0/minicluster-data/master-0/data/info.pb
I20260806 18:34:31.668533 25657 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 18:34:31.673666 25657 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 84961f79d4ef4ab1afb11a93d69ffae5: Bootstrap starting.
I20260806 18:34:31.675045 20232 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJW69tn/build/debug/bin/kudu as pid 25589
I20260806 18:34:31.675173 20232 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786041246094603-20232-0/minicluster-data/master-0/wal/instance
I20260806 18:34:31.675484 20232 external_mini_cluster.cc:1428] Setting key 2ac48a8f3a3a9b05a5805eca265f74c2
I20260806 18:34:31.676343 25657 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 84961f79d4ef4ab1afb11a93d69ffae5: Neither blocks nor log segments found. Creating new log.
I20260806 18:34:31.677141 25657 log.cc:824] T 00000000000000000000000000000000 P 84961f79d4ef4ab1afb11a93d69ffae5: Log is configured to *not* fsync() on all Append() calls
I20260806 18:34:31.679284 25657 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 84961f79d4ef4ab1afb11a93d69ffae5: No bootstrap required, opened a new log
I20260806 18:34:31.683475 25657 raft_consensus.cc:348] T 00000000000000000000000000000000 P 84961f79d4ef4ab1afb11a93d69ffae5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "84961f79d4ef4ab1afb11a93d69ffae5" member_type: VOTER last_known_addr { host: "127.19.194.62" port: 40233 } }
I20260806 18:34:31.683779 25657 raft_consensus.cc:374] T 00000000000000000000000000000000 P 84961f79d4ef4ab1afb11a93d69ffae5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 18:34:31.683859 25657 raft_consensus.cc:729] T 00000000000000000000000000000000 P 84961f79d4ef4ab1afb11a93d69ffae5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 84961f79d4ef4ab1afb11a93d69ffae5, State: Initialized, Role: FOLLOWER
I20260806 18:34:31.684327 25657 consensus_queue.cc:260] T 00000000000000000000000000000000 P 84961f79d4ef4ab1afb11a93d69ffae5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "84961f79d4ef4ab1afb11a93d69ffae5" member_type: VOTER last_known_addr { host: "127.19.194.62" port: 40233 } }
I20260806 18:34:31.684446 25657 raft_consensus.cc:388] T 00000000000000000000000000000000 P 84961f79d4ef4ab1afb11a93d69ffae5 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 18:34:31.684532 25657 raft_consensus.cc:482] T 00000000000000000000000000000000 P 84961f79d4ef4ab1afb11a93d69ffae5 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 18:34:31.684676 25657 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 84961f79d4ef4ab1afb11a93d69ffae5 [term 0 FOLLOWER]: Advancing to term 1
I20260806 18:34:31.684791 25660 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:34:31.676579 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39906 (local address 127.19.194.62:40233)
0806 18:34:31.677178 (+ 599us) server_negotiation.cc:206] Beginning negotiation
0806 18:34:31.677191 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0806 18:34:31.677226 (+ 35us) server_negotiation.cc:407] Connection header received
0806 18:34:31.678104 (+ 878us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:34:31.678124 (+ 20us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:34:31.678443 (+ 319us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:34:31.678708 (+ 265us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:34:31.679071 (+ 363us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:34:31.680100 (+ 1029us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:34:31.681344 (+ 1244us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:34:31.681816 (+ 472us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:34:31.681934 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:34:31.682804 (+ 870us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:34:31.682809 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:34:31.682821 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 18:34:31.683270 (+ 449us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:34:31.683325 (+ 55us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 18:34:31.683345 (+ 20us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:34:31.683420 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0806 18:34:31.683640 (+ 220us) server_negotiation.cc:299] Negotiation successful
0806 18:34:31.683937 (+ 297us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":241,"thread_start_us":123,"threads_started":1}
I20260806 18:34:31.685559 25657 raft_consensus.cc:504] T 00000000000000000000000000000000 P 84961f79d4ef4ab1afb11a93d69ffae5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "84961f79d4ef4ab1afb11a93d69ffae5" member_type: VOTER last_known_addr { host: "127.19.194.62" port: 40233 } }
I20260806 18:34:31.685886 25657 leader_election.cc:302] T 00000000000000000000000000000000 P 84961f79d4ef4ab1afb11a93d69ffae5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 84961f79d4ef4ab1afb11a93d69ffae5; no voters:
I20260806 18:34:31.686210 25657 leader_election.cc:288] T 00000000000000000000000000000000 P 84961f79d4ef4ab1afb11a93d69ffae5 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 18:34:31.686295 25662 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 84961f79d4ef4ab1afb11a93d69ffae5 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 18:34:31.686518 25662 raft_consensus.cc:686] T 00000000000000000000000000000000 P 84961f79d4ef4ab1afb11a93d69ffae5 [term 1 LEADER]: Becoming Leader. State: Replica: 84961f79d4ef4ab1afb11a93d69ffae5, State: Running, Role: LEADER
I20260806 18:34:31.686803 25662 consensus_queue.cc:237] T 00000000000000000000000000000000 P 84961f79d4ef4ab1afb11a93d69ffae5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "84961f79d4ef4ab1afb11a93d69ffae5" member_type: VOTER last_known_addr { host: "127.19.194.62" port: 40233 } }
I20260806 18:34:31.687492 25657 sys_catalog.cc:564] T 00000000000000000000000000000000 P 84961f79d4ef4ab1afb11a93d69ffae5 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 18:34:31.687532 20232 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260806 18:34:31.688331 25664 sys_catalog.cc:455] T 00000000000000000000000000000000 P 84961f79d4ef4ab1afb11a93d69ffae5 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 84961f79d4ef4ab1afb11a93d69ffae5. Latest consensus state: current_term: 1 leader_uuid: "84961f79d4ef4ab1afb11a93d69ffae5" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "84961f79d4ef4ab1afb11a93d69ffae5" member_type: VOTER last_known_addr { host: "127.19.194.62" port: 40233 } } }
I20260806 18:34:31.688485 25664 sys_catalog.cc:458] T 00000000000000000000000000000000 P 84961f79d4ef4ab1afb11a93d69ffae5 [sys.catalog]: This master's current role is: LEADER
I20260806 18:34:31.688730 25664 sys_catalog.cc:455] T 00000000000000000000000000000000 P 84961f79d4ef4ab1afb11a93d69ffae5 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "84961f79d4ef4ab1afb11a93d69ffae5" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "84961f79d4ef4ab1afb11a93d69ffae5" member_type: VOTER last_known_addr { host: "127.19.194.62" port: 40233 } } }
I20260806 18:34:31.688824 25664 sys_catalog.cc:458] T 00000000000000000000000000000000 P 84961f79d4ef4ab1afb11a93d69ffae5 [sys.catalog]: This master's current role is: LEADER
I20260806 18:34:31.690047 25671 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 18:34:31.692921 25671 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 18:34:31.698144 25671 catalog_manager.cc:1347] Generated new cluster ID: 6f90203fc45446928e006a66e28e2137
I20260806 18:34:31.698210 25671 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 18:34:31.719127 25671 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 18:34:31.720114 25671 catalog_manager.cc:1504] Loading token signing keys...
I20260806 18:34:31.729907 25671 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 84961f79d4ef4ab1afb11a93d69ffae5: Generated new TSK 0
I20260806 18:34:31.730664 25671 catalog_manager.cc:1514] Initializing in-progress tserver states...
E20260806 18:34:31.764883 25688 webserver.cc:533] Webserver: sslize failed: error:14094415:SSL routines:ssl3_read_bytes:sslv3 alert certificate expired
E20260806 18:34:31.765031 25688 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
I20260806 18:34:31.765618 25660 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:34:31.748981 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39910 (local address 127.19.194.62:40233)
0806 18:34:31.749122 (+ 141us) server_negotiation.cc:206] Beginning negotiation
0806 18:34:31.749126 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 18:34:31.749338 (+ 212us) server_negotiation.cc:407] Connection header received
0806 18:34:31.749515 (+ 177us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:34:31.749518 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:34:31.749571 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:34:31.749671 (+ 100us) server_negotiation.cc:226] Negotiated authn=JWT
0806 18:34:31.749898 (+ 227us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:34:31.750698 (+ 800us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:34:31.751536 (+ 838us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:34:31.751866 (+ 330us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:34:31.751968 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:34:31.752251 (+ 283us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0806 18:34:31.765239 (+ 12988us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:36213/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
0806 18:34:31.765486 (+ 247us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:39910: Failed to load JWKS: Error downloading JWKS from 'https://localhost:36213/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Metrics: {"server-negotiator.queue_time_us":62}
W20260806 18:34:31.765697 25660 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:39910: Failed to load JWKS: Error downloading JWKS from 'https://localhost:36213/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
W20260806 18:34:31.766703 25687 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 18:34:31.748915 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.194.62:40233 (local address 127.0.0.1:39910)
0806 18:34:31.749264 (+ 349us) negotiation.cc:107] Waiting for socket to connect
0806 18:34:31.749275 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0806 18:34:31.749373 (+ 98us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:34:31.749691 (+ 318us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:34:31.749693 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:34:31.749699 (+ 6us) client_negotiation.cc:189] Negotiated authn=JWT
0806 18:34:31.749799 (+ 100us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:34:31.749803 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:34:31.750798 (+ 995us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:34:31.750800 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:34:31.751416 (+ 616us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:34:31.751425 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:34:31.751978 (+ 553us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:34:31.751980 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:34:31.752050 (+ 70us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:34:31.752089 (+ 39us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0806 18:34:31.765428 (+ 13339us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:36213/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
0806 18:34:31.765604 (+ 176us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.19.194.62:40233: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:36213/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Metrics: {"client-negotiator.queue_time_us":241,"thread_start_us":138,"threads_started":1}
/data0/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/integration-tests/security-itest.cc:597: Failure
Failed
Bad status: Runtime error: Could not connect to the cluster: Client connection negotiation failed: client connection to 127.19.194.62:40233: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:36213/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Google Test trace:
/data0/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/integration-tests/security-itest.cc:593: Valid JWT
I20260806 18:34:31.767751 20232 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJW69tn/build/debug/bin/kudu with pid 25589
2026-08-06T18:34:31Z chronyd exiting
I20260806 18:34:31.807076 20232 test_util.cc:175] -----------------------------------------------
I20260806 18:34:31.807233 20232 test_util.cc:176] Had fatal failures, leaving test files at /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786041246094603-20232-0
[ FAILED ] SecurityITest.TestJwtMiniCluster (330 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithInvalidCert
I20260806 18:34:31.808518 20232 mini_oidc.cc:152] Starting JWKS server
I20260806 18:34:31.809418 20232 webserver.cc:460] Webserver started at https://127.0.0.1:35533/ using document root <none> and password file <none>
I20260806 18:34:31.809543 20232 mini_oidc.cc:189] Starting OIDC Discovery server
I20260806 18:34:31.809814 20232 webserver.cc:460] Webserver started at http://127.0.0.1:33725/ using document root <none> and password file <none>
2026-08-06T18:34:31Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T18:34:31Z Disabled control of system clock
I20260806 18:34:31.819271 20232 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJW69tn/build/debug/bin/kudu
/tmp/dist-test-taskJW69tn/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786041246094603-20232-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786041246094603-20232-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786041246094603-20232-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786041246094603-20232-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.19.194.62:32909
--webserver_interface=127.19.194.62
--webserver_port=0
--builtin_ntp_servers=127.19.194.20:44217
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.19.194.62:32909
--enable_jwt_token_auth=true
--jwks_url=https://localhost:35533/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786041246094603-20232-0/testchainca.cert with env {}
W20260806 18:34:31.925886 25695 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 18:34:31.926111 25695 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 18:34:31.926178 25695 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 18:34:31.930153 25695 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 18:34:31.930205 25695 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 18:34:31.930222 25695 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 18:34:31.930238 25695 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 18:34:31.930253 25695 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260806 18:34:31.930318 25695 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260806 18:34:31.930346 25695 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:35533/jwks.json
I20260806 18:34:31.935498 25695 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.194.20:44217
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786041246094603-20232-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786041246094603-20232-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.19.194.62:32909
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.19.194.62:32909
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:35533/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786041246094603-20232-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.19.194.62
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786041246094603-20232-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25695
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786041246094603-20232-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 17:59:13 UTC on ccycloud.quasar-wvsnzx.root.comops.site
build id 4571028
I20260806 18:34:31.936406 25695 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 18:34:31.937307 25695 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 18:34:31.945477 25701 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:34:31.945508 25704 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 18:34:31.945619 25695 server_base.cc:1034] running on GCE node
W20260806 18:34:31.945539 25702 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 18:34:31.946347 25695 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 18:34:31.947398 25695 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 18:34:31.948582 25695 hybrid_clock.cc:648] HybridClock initialized: now 1786041271948558 us; error 39 us; skew 500 ppm
I20260806 18:34:31.950263 25695 webserver.cc:460] Webserver started at http://127.19.194.62:37423/ using document root <none> and password file <none>
I20260806 18:34:31.950790 25695 fs_manager.cc:359] Metadata directory not provided
I20260806 18:34:31.950877 25695 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 18:34:31.951097 25695 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 18:34:31.953045 25695 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786041246094603-20232-0/minicluster-data/master-0/data/instance:
uuid: "4387de9704eb4afeb31e568bc3480bed"
format_stamp: "Formatted at 2026-08-06 18:34:31 on dist-test-slave-1xrh"
server_key: "a923c8205c47777bcafa2d73193b6179"
server_key_iv: "68d2b1e8ebe17abd993891c836f025ae"
server_key_version: "encryptionkey@0"
I20260806 18:34:31.953589 25695 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786041246094603-20232-0/minicluster-data/master-0/wal/instance:
uuid: "4387de9704eb4afeb31e568bc3480bed"
format_stamp: "Formatted at 2026-08-06 18:34:31 on dist-test-slave-1xrh"
server_key: "a923c8205c47777bcafa2d73193b6179"
server_key_iv: "68d2b1e8ebe17abd993891c836f025ae"
server_key_version: "encryptionkey@0"
I20260806 18:34:31.957099 25695 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.005s sys 0.000s
I20260806 18:34:31.959368 25710 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 18:34:31.960434 25695 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260806 18:34:31.960589 25695 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786041246094603-20232-0/minicluster-data/master-0/data,/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786041246094603-20232-0/minicluster-data/master-0/wal
uuid: "4387de9704eb4afeb31e568bc3480bed"
format_stamp: "Formatted at 2026-08-06 18:34:31 on dist-test-slave-1xrh"
server_key: "a923c8205c47777bcafa2d73193b6179"
server_key_iv: "68d2b1e8ebe17abd993891c836f025ae"
server_key_version: "encryptionkey@0"
I20260806 18:34:31.960698 25695 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786041246094603-20232-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786041246094603-20232-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786041246094603-20232-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 18:34:31.973202 25695 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 18:34:31.973809 25695 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 18:34:31.973997 25695 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 18:34:31.981129 25764 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.194.62:32909 every 8 connection(s)
I20260806 18:34:31.981140 25695 rpc_server.cc:307] RPC server started. Bound to: 127.19.194.62:32909
I20260806 18:34:31.982455 25695 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786041246094603-20232-0/minicluster-data/master-0/data/info.pb
I20260806 18:34:31.984901 20232 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJW69tn/build/debug/bin/kudu as pid 25695
I20260806 18:34:31.985041 20232 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786041246094603-20232-0/minicluster-data/master-0/wal/instance
I20260806 18:34:31.985389 20232 external_mini_cluster.cc:1428] Setting key 8309e20a766d5d51e0d0075933114b53
I20260806 18:34:31.985788 25765 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 18:34:31.992866 25765 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4387de9704eb4afeb31e568bc3480bed: Bootstrap starting.
I20260806 18:34:31.994526 25768 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:34:31.986754 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60572 (local address 127.19.194.62:32909)
0806 18:34:31.987430 (+ 676us) server_negotiation.cc:206] Beginning negotiation
0806 18:34:31.987446 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0806 18:34:31.987483 (+ 37us) server_negotiation.cc:407] Connection header received
0806 18:34:31.988149 (+ 666us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:34:31.988165 (+ 16us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:34:31.988548 (+ 383us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:34:31.988811 (+ 263us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:34:31.989135 (+ 324us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:34:31.990072 (+ 937us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:34:31.991288 (+ 1216us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:34:31.991773 (+ 485us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:34:31.991887 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:34:31.992802 (+ 915us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:34:31.992807 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:34:31.992819 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 18:34:31.993254 (+ 435us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:34:31.993310 (+ 56us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 18:34:31.993331 (+ 21us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:34:31.993407 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0806 18:34:31.993564 (+ 157us) server_negotiation.cc:299] Negotiation successful
0806 18:34:31.993848 (+ 284us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":298,"thread_start_us":146,"threads_started":1}
I20260806 18:34:31.995278 25765 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 4387de9704eb4afeb31e568bc3480bed: Neither blocks nor log segments found. Creating new log.
I20260806 18:34:31.996179 25765 log.cc:824] T 00000000000000000000000000000000 P 4387de9704eb4afeb31e568bc3480bed: Log is configured to *not* fsync() on all Append() calls
I20260806 18:34:31.997537 20232 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260806 18:34:31.998489 25765 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4387de9704eb4afeb31e568bc3480bed: No bootstrap required, opened a new log
I20260806 18:34:32.002169 25765 raft_consensus.cc:348] T 00000000000000000000000000000000 P 4387de9704eb4afeb31e568bc3480bed [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4387de9704eb4afeb31e568bc3480bed" member_type: VOTER last_known_addr { host: "127.19.194.62" port: 32909 } }
I20260806 18:34:32.002369 25765 raft_consensus.cc:374] T 00000000000000000000000000000000 P 4387de9704eb4afeb31e568bc3480bed [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 18:34:32.002410 25765 raft_consensus.cc:729] T 00000000000000000000000000000000 P 4387de9704eb4afeb31e568bc3480bed [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4387de9704eb4afeb31e568bc3480bed, State: Initialized, Role: FOLLOWER
I20260806 18:34:32.002748 25765 consensus_queue.cc:260] T 00000000000000000000000000000000 P 4387de9704eb4afeb31e568bc3480bed [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4387de9704eb4afeb31e568bc3480bed" member_type: VOTER last_known_addr { host: "127.19.194.62" port: 32909 } }
I20260806 18:34:32.002856 25765 raft_consensus.cc:388] T 00000000000000000000000000000000 P 4387de9704eb4afeb31e568bc3480bed [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 18:34:32.002899 25765 raft_consensus.cc:482] T 00000000000000000000000000000000 P 4387de9704eb4afeb31e568bc3480bed [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 18:34:32.002965 25765 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 4387de9704eb4afeb31e568bc3480bed [term 0 FOLLOWER]: Advancing to term 1
I20260806 18:34:32.003875 25765 raft_consensus.cc:504] T 00000000000000000000000000000000 P 4387de9704eb4afeb31e568bc3480bed [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4387de9704eb4afeb31e568bc3480bed" member_type: VOTER last_known_addr { host: "127.19.194.62" port: 32909 } }
I20260806 18:34:32.004161 25765 leader_election.cc:302] T 00000000000000000000000000000000 P 4387de9704eb4afeb31e568bc3480bed [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 4387de9704eb4afeb31e568bc3480bed; no voters:
I20260806 18:34:32.004401 25765 leader_election.cc:288] T 00000000000000000000000000000000 P 4387de9704eb4afeb31e568bc3480bed [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 18:34:32.004522 25770 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 4387de9704eb4afeb31e568bc3480bed [term 1 FOLLOWER]: Leader election won for term 1
I20260806 18:34:32.004784 25770 raft_consensus.cc:686] T 00000000000000000000000000000000 P 4387de9704eb4afeb31e568bc3480bed [term 1 LEADER]: Becoming Leader. State: Replica: 4387de9704eb4afeb31e568bc3480bed, State: Running, Role: LEADER
I20260806 18:34:32.005108 25770 consensus_queue.cc:237] T 00000000000000000000000000000000 P 4387de9704eb4afeb31e568bc3480bed [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4387de9704eb4afeb31e568bc3480bed" member_type: VOTER last_known_addr { host: "127.19.194.62" port: 32909 } }
I20260806 18:34:32.005645 25765 sys_catalog.cc:564] T 00000000000000000000000000000000 P 4387de9704eb4afeb31e568bc3480bed [sys.catalog]: configured and running, proceeding with master startup.
I20260806 18:34:32.006976 25772 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4387de9704eb4afeb31e568bc3480bed [sys.catalog]: SysCatalogTable state changed. Reason: New leader 4387de9704eb4afeb31e568bc3480bed. Latest consensus state: current_term: 1 leader_uuid: "4387de9704eb4afeb31e568bc3480bed" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4387de9704eb4afeb31e568bc3480bed" member_type: VOTER last_known_addr { host: "127.19.194.62" port: 32909 } } }
I20260806 18:34:32.007103 25772 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4387de9704eb4afeb31e568bc3480bed [sys.catalog]: This master's current role is: LEADER
I20260806 18:34:32.007393 25771 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4387de9704eb4afeb31e568bc3480bed [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "4387de9704eb4afeb31e568bc3480bed" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4387de9704eb4afeb31e568bc3480bed" member_type: VOTER last_known_addr { host: "127.19.194.62" port: 32909 } } }
I20260806 18:34:32.007485 25771 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4387de9704eb4afeb31e568bc3480bed [sys.catalog]: This master's current role is: LEADER
I20260806 18:34:32.007644 25779 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 18:34:32.010553 25779 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 18:34:32.015525 25779 catalog_manager.cc:1347] Generated new cluster ID: 48cb2219654349478b7066c2ab7828b6
I20260806 18:34:32.015589 25779 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 18:34:32.033960 25779 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 18:34:32.035082 25779 catalog_manager.cc:1504] Loading token signing keys...
I20260806 18:34:32.042335 25779 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 4387de9704eb4afeb31e568bc3480bed: Generated new TSK 0
I20260806 18:34:32.043044 25779 catalog_manager.cc:1514] Initializing in-progress tserver states...
E20260806 18:34:32.075852 25795 webserver.cc:533] Webserver: sslize failed: error:14094418:SSL routines:ssl3_read_bytes:tlsv1 alert unknown ca
E20260806 18:34:32.075950 25795 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
I20260806 18:34:32.076633 25768 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:34:32.056806 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60588 (local address 127.19.194.62:32909)
0806 18:34:32.056937 (+ 131us) server_negotiation.cc:206] Beginning negotiation
0806 18:34:32.056942 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 18:34:32.057151 (+ 209us) server_negotiation.cc:407] Connection header received
0806 18:34:32.057295 (+ 144us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:34:32.057297 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:34:32.057341 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:34:32.057421 (+ 80us) server_negotiation.cc:226] Negotiated authn=JWT
0806 18:34:32.057677 (+ 256us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:34:32.058169 (+ 492us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:34:32.059295 (+ 1126us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:34:32.059486 (+ 191us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:34:32.059560 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:34:32.059868 (+ 308us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0806 18:34:32.076231 (+ 16363us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:35533/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
0806 18:34:32.076499 (+ 268us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:60588: Failed to load JWKS: Error downloading JWKS from 'https://localhost:35533/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
Metrics: {"server-negotiator.queue_time_us":49}
W20260806 18:34:32.076697 25768 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:60588: Failed to load JWKS: Error downloading JWKS from 'https://localhost:35533/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
W20260806 18:34:32.076723 25794 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 18:34:32.056755 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.194.62:32909 (local address 127.0.0.1:60588)
0806 18:34:32.057060 (+ 305us) negotiation.cc:107] Waiting for socket to connect
0806 18:34:32.057076 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0806 18:34:32.057185 (+ 109us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:34:32.057447 (+ 262us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:34:32.057450 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:34:32.057458 (+ 8us) client_negotiation.cc:189] Negotiated authn=JWT
0806 18:34:32.057576 (+ 118us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:34:32.057583 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:34:32.058271 (+ 688us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:34:32.058274 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:34:32.059192 (+ 918us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:34:32.059203 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:34:32.059590 (+ 387us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:34:32.059594 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:34:32.059695 (+ 101us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:34:32.059721 (+ 26us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0806 18:34:32.076419 (+ 16698us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:35533/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
0806 18:34:32.076563 (+ 144us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.19.194.62:32909: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:35533/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
Metrics: {"client-negotiator.queue_time_us":222,"thread_start_us":121,"threads_started":1}
I20260806 18:34:32.077368 20232 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJW69tn/build/debug/bin/kudu with pid 25695
2026-08-06T18:34:32Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithInvalidCert (309 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithUntrustedCert
I20260806 18:34:32.118175 20232 mini_oidc.cc:152] Starting JWKS server
I20260806 18:34:32.119011 20232 webserver.cc:460] Webserver started at https://127.0.0.1:43361/ using document root <none> and password file <none>
I20260806 18:34:32.119184 20232 mini_oidc.cc:189] Starting OIDC Discovery server
I20260806 18:34:32.119521 20232 webserver.cc:460] Webserver started at http://127.0.0.1:33901/ using document root <none> and password file <none>
2026-08-06T18:34:32Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T18:34:32Z Disabled control of system clock
I20260806 18:34:32.128762 20232 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJW69tn/build/debug/bin/kudu
/tmp/dist-test-taskJW69tn/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786041246094603-20232-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786041246094603-20232-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786041246094603-20232-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786041246094603-20232-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.19.194.62:44393
--webserver_interface=127.19.194.62
--webserver_port=0
--builtin_ntp_servers=127.19.194.20:42461
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.19.194.62:44393
--enable_jwt_token_auth=true
--jwks_url=https://localhost:43361/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {}
W20260806 18:34:32.236541 25802 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 18:34:32.236770 25802 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 18:34:32.236812 25802 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 18:34:32.240562 25802 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 18:34:32.240605 25802 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 18:34:32.240619 25802 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 18:34:32.240635 25802 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 18:34:32.240649 25802 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260806 18:34:32.240664 25802 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260806 18:34:32.240676 25802 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:43361/jwks.json
I20260806 18:34:32.245213 25802 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.194.20:42461
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786041246094603-20232-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786041246094603-20232-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.19.194.62:44393
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.19.194.62:44393
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:43361/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786041246094603-20232-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.19.194.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25802
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786041246094603-20232-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 17:59:13 UTC on ccycloud.quasar-wvsnzx.root.comops.site
build id 4571028
I20260806 18:34:32.246109 25802 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 18:34:32.247399 25802 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 18:34:32.255304 25810 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:34:32.255344 25808 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:34:32.255344 25807 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 18:34:32.255458 25802 server_base.cc:1034] running on GCE node
I20260806 18:34:32.256188 25802 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 18:34:32.257153 25802 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 18:34:32.258327 25802 hybrid_clock.cc:648] HybridClock initialized: now 1786041272258299 us; error 36 us; skew 500 ppm
I20260806 18:34:32.260171 25802 webserver.cc:460] Webserver started at http://127.19.194.62:44573/ using document root <none> and password file <none>
I20260806 18:34:32.260707 25802 fs_manager.cc:359] Metadata directory not provided
I20260806 18:34:32.260769 25802 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 18:34:32.261035 25802 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 18:34:32.262964 25802 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786041246094603-20232-0/minicluster-data/master-0/data/instance:
uuid: "8780471dc9b2467ea27b1a044ae5a587"
format_stamp: "Formatted at 2026-08-06 18:34:32 on dist-test-slave-1xrh"
server_key: "d92bbef19edadc5ea9938f33817edff4"
server_key_iv: "a62c82178cae9e437ab88aa64411d0c7"
server_key_version: "encryptionkey@0"
I20260806 18:34:32.263571 25802 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786041246094603-20232-0/minicluster-data/master-0/wal/instance:
uuid: "8780471dc9b2467ea27b1a044ae5a587"
format_stamp: "Formatted at 2026-08-06 18:34:32 on dist-test-slave-1xrh"
server_key: "d92bbef19edadc5ea9938f33817edff4"
server_key_iv: "a62c82178cae9e437ab88aa64411d0c7"
server_key_version: "encryptionkey@0"
I20260806 18:34:32.267330 25802 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.001s
I20260806 18:34:32.269649 25816 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 18:34:32.270798 25802 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260806 18:34:32.270942 25802 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786041246094603-20232-0/minicluster-data/master-0/data,/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786041246094603-20232-0/minicluster-data/master-0/wal
uuid: "8780471dc9b2467ea27b1a044ae5a587"
format_stamp: "Formatted at 2026-08-06 18:34:32 on dist-test-slave-1xrh"
server_key: "d92bbef19edadc5ea9938f33817edff4"
server_key_iv: "a62c82178cae9e437ab88aa64411d0c7"
server_key_version: "encryptionkey@0"
I20260806 18:34:32.271050 25802 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786041246094603-20232-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786041246094603-20232-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786041246094603-20232-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 18:34:32.287257 25802 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 18:34:32.287842 25802 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 18:34:32.288028 25802 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 18:34:32.294751 25802 rpc_server.cc:307] RPC server started. Bound to: 127.19.194.62:44393
I20260806 18:34:32.294762 25870 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.194.62:44393 every 8 connection(s)
I20260806 18:34:32.295632 25802 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786041246094603-20232-0/minicluster-data/master-0/data/info.pb
I20260806 18:34:32.298367 25871 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 18:34:32.303511 25871 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8780471dc9b2467ea27b1a044ae5a587: Bootstrap starting.
I20260806 18:34:32.304108 20232 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJW69tn/build/debug/bin/kudu as pid 25802
I20260806 18:34:32.304244 20232 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786041246094603-20232-0/minicluster-data/master-0/wal/instance
I20260806 18:34:32.304527 20232 external_mini_cluster.cc:1428] Setting key f30194dbb4f0f67483b9a519ab54f5de
I20260806 18:34:32.306186 25871 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 8780471dc9b2467ea27b1a044ae5a587: Neither blocks nor log segments found. Creating new log.
I20260806 18:34:32.306919 25871 log.cc:824] T 00000000000000000000000000000000 P 8780471dc9b2467ea27b1a044ae5a587: Log is configured to *not* fsync() on all Append() calls
I20260806 18:34:32.308683 25871 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8780471dc9b2467ea27b1a044ae5a587: No bootstrap required, opened a new log
I20260806 18:34:32.312167 25871 raft_consensus.cc:348] T 00000000000000000000000000000000 P 8780471dc9b2467ea27b1a044ae5a587 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8780471dc9b2467ea27b1a044ae5a587" member_type: VOTER last_known_addr { host: "127.19.194.62" port: 44393 } }
I20260806 18:34:32.312392 25871 raft_consensus.cc:374] T 00000000000000000000000000000000 P 8780471dc9b2467ea27b1a044ae5a587 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 18:34:32.312453 25871 raft_consensus.cc:729] T 00000000000000000000000000000000 P 8780471dc9b2467ea27b1a044ae5a587 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8780471dc9b2467ea27b1a044ae5a587, State: Initialized, Role: FOLLOWER
I20260806 18:34:32.312886 25871 consensus_queue.cc:260] T 00000000000000000000000000000000 P 8780471dc9b2467ea27b1a044ae5a587 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8780471dc9b2467ea27b1a044ae5a587" member_type: VOTER last_known_addr { host: "127.19.194.62" port: 44393 } }
I20260806 18:34:32.313035 25871 raft_consensus.cc:388] T 00000000000000000000000000000000 P 8780471dc9b2467ea27b1a044ae5a587 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 18:34:32.313122 25871 raft_consensus.cc:482] T 00000000000000000000000000000000 P 8780471dc9b2467ea27b1a044ae5a587 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 18:34:32.313213 25871 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 8780471dc9b2467ea27b1a044ae5a587 [term 0 FOLLOWER]: Advancing to term 1
I20260806 18:34:32.313364 25874 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:34:32.305669 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38076 (local address 127.19.194.62:44393)
0806 18:34:32.306265 (+ 596us) server_negotiation.cc:206] Beginning negotiation
0806 18:34:32.306279 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0806 18:34:32.306363 (+ 84us) server_negotiation.cc:407] Connection header received
0806 18:34:32.306941 (+ 578us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:34:32.306956 (+ 15us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:34:32.307425 (+ 469us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:34:32.307728 (+ 303us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:34:32.308123 (+ 395us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:34:32.309041 (+ 918us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:34:32.310110 (+ 1069us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:34:32.310536 (+ 426us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:34:32.310697 (+ 161us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:34:32.311629 (+ 932us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:34:32.311638 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:34:32.311649 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 18:34:32.311997 (+ 348us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:34:32.312045 (+ 48us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 18:34:32.312063 (+ 18us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:34:32.312166 (+ 103us) server_negotiation.cc:1035] Waiting for connection context
0806 18:34:32.312309 (+ 143us) server_negotiation.cc:299] Negotiation successful
0806 18:34:32.312560 (+ 251us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":274,"thread_start_us":118,"threads_started":1}
I20260806 18:34:32.314334 25871 raft_consensus.cc:504] T 00000000000000000000000000000000 P 8780471dc9b2467ea27b1a044ae5a587 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8780471dc9b2467ea27b1a044ae5a587" member_type: VOTER last_known_addr { host: "127.19.194.62" port: 44393 } }
I20260806 18:34:32.314661 25871 leader_election.cc:302] T 00000000000000000000000000000000 P 8780471dc9b2467ea27b1a044ae5a587 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 8780471dc9b2467ea27b1a044ae5a587; no voters:
I20260806 18:34:32.314994 25871 leader_election.cc:288] T 00000000000000000000000000000000 P 8780471dc9b2467ea27b1a044ae5a587 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 18:34:32.315062 25876 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 8780471dc9b2467ea27b1a044ae5a587 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 18:34:32.315583 25876 raft_consensus.cc:686] T 00000000000000000000000000000000 P 8780471dc9b2467ea27b1a044ae5a587 [term 1 LEADER]: Becoming Leader. State: Replica: 8780471dc9b2467ea27b1a044ae5a587, State: Running, Role: LEADER
I20260806 18:34:32.315863 25876 consensus_queue.cc:237] T 00000000000000000000000000000000 P 8780471dc9b2467ea27b1a044ae5a587 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8780471dc9b2467ea27b1a044ae5a587" member_type: VOTER last_known_addr { host: "127.19.194.62" port: 44393 } }
I20260806 18:34:32.316246 25871 sys_catalog.cc:564] T 00000000000000000000000000000000 P 8780471dc9b2467ea27b1a044ae5a587 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 18:34:32.316395 20232 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260806 18:34:32.317039 25876 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8780471dc9b2467ea27b1a044ae5a587 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 8780471dc9b2467ea27b1a044ae5a587. Latest consensus state: current_term: 1 leader_uuid: "8780471dc9b2467ea27b1a044ae5a587" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8780471dc9b2467ea27b1a044ae5a587" member_type: VOTER last_known_addr { host: "127.19.194.62" port: 44393 } } }
I20260806 18:34:32.317178 25876 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8780471dc9b2467ea27b1a044ae5a587 [sys.catalog]: This master's current role is: LEADER
I20260806 18:34:32.318032 25877 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8780471dc9b2467ea27b1a044ae5a587 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "8780471dc9b2467ea27b1a044ae5a587" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8780471dc9b2467ea27b1a044ae5a587" member_type: VOTER last_known_addr { host: "127.19.194.62" port: 44393 } } }
I20260806 18:34:32.318135 25877 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8780471dc9b2467ea27b1a044ae5a587 [sys.catalog]: This master's current role is: LEADER
I20260806 18:34:32.318450 25885 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 18:34:32.321053 25885 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 18:34:32.326273 25885 catalog_manager.cc:1347] Generated new cluster ID: 750caed74e5d4a349f086c56ccad8406
I20260806 18:34:32.326334 25885 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 18:34:32.336917 25885 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 18:34:32.337741 25885 catalog_manager.cc:1504] Loading token signing keys...
I20260806 18:34:32.342520 25885 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 8780471dc9b2467ea27b1a044ae5a587: Generated new TSK 0
I20260806 18:34:32.343106 25885 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 18:34:32.357893 25874 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:34:32.352890 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38086 (local address 127.19.194.62:44393)
0806 18:34:32.353012 (+ 122us) server_negotiation.cc:206] Beginning negotiation
0806 18:34:32.353017 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 18:34:32.353159 (+ 142us) server_negotiation.cc:407] Connection header received
0806 18:34:32.353305 (+ 146us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:34:32.353308 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:34:32.353365 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:34:32.353478 (+ 113us) server_negotiation.cc:226] Negotiated authn=JWT
0806 18:34:32.353677 (+ 199us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:34:32.354453 (+ 776us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:34:32.355413 (+ 960us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:34:32.355776 (+ 363us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:34:32.355881 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:34:32.356139 (+ 258us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0806 18:34:32.357494 (+ 1355us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:43361/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
0806 18:34:32.357729 (+ 235us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:38086: Failed to load JWKS: Error downloading JWKS from 'https://localhost:43361/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
Metrics: {"server-negotiator.queue_time_us":47}
W20260806 18:34:32.357959 25874 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:38086: Failed to load JWKS: Error downloading JWKS from 'https://localhost:43361/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
W20260806 18:34:32.359588 25900 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 18:34:32.352773 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.194.62:44393 (local address 127.0.0.1:38086)
0806 18:34:32.353077 (+ 304us) negotiation.cc:107] Waiting for socket to connect
0806 18:34:32.353088 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0806 18:34:32.353181 (+ 93us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:34:32.353475 (+ 294us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:34:32.353476 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:34:32.353482 (+ 6us) client_negotiation.cc:189] Negotiated authn=JWT
0806 18:34:32.353582 (+ 100us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:34:32.353586 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:34:32.354585 (+ 999us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:34:32.354587 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:34:32.355304 (+ 717us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:34:32.355313 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:34:32.355896 (+ 583us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:34:32.355899 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:34:32.355960 (+ 61us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:34:32.355980 (+ 20us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0806 18:34:32.357665 (+ 1685us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:43361/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
0806 18:34:32.359427 (+ 1762us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.19.194.62:44393: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:43361/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
Metrics: {"client-negotiator.queue_time_us":221,"mutex_wait_us":1584,"thread_start_us":108,"threads_started":1}
I20260806 18:34:32.360085 20232 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJW69tn/build/debug/bin/kudu with pid 25802
E20260806 18:34:32.369647 25901 webserver.cc:533] Webserver: sslize failed:
E20260806 18:34:32.369709 25901 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
2026-08-06T18:34:32Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithUntrustedCert (283 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS
2026-08-06T18:34:32Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T18:34:32Z Disabled control of system clock
I20260806 18:34:32.410854 20232 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJW69tn/build/debug/bin/kudu
/tmp/dist-test-taskJW69tn/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786041246094603-20232-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786041246094603-20232-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786041246094603-20232-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786041246094603-20232-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.19.194.62:39547
--webserver_interface=127.19.194.62
--webserver_port=0
--builtin_ntp_servers=127.19.194.20:36559
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.19.194.62:39547
--enable_jwt_token_auth=true
--jwks_url=default.url
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786041246094603-20232-0/testchainca.cert with env {}
W20260806 18:34:32.517465 25905 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 18:34:32.517724 25905 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 18:34:32.517797 25905 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 18:34:32.521632 25905 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 18:34:32.521711 25905 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 18:34:32.521739 25905 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 18:34:32.521777 25905 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 18:34:32.521821 25905 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260806 18:34:32.521855 25905 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260806 18:34:32.521883 25905 flags.cc:425] Enabled experimental flag: --jwks_url=default.url
I20260806 18:34:32.526791 25905 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.194.20:36559
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786041246094603-20232-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786041246094603-20232-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.19.194.62:39547
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.19.194.62:39547
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=default.url
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786041246094603-20232-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.19.194.62
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786041246094603-20232-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25905
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786041246094603-20232-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 17:59:13 UTC on ccycloud.quasar-wvsnzx.root.comops.site
build id 4571028
I20260806 18:34:32.527782 25905 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 18:34:32.528693 25905 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 18:34:32.536716 25910 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 18:34:32.536895 25905 server_base.cc:1034] running on GCE node
W20260806 18:34:32.536711 25913 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:34:32.536752 25911 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 18:34:32.537549 25905 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 18:34:32.538546 25905 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 18:34:32.539742 25905 hybrid_clock.cc:648] HybridClock initialized: now 1786041272539725 us; error 39 us; skew 500 ppm
I20260806 18:34:32.541389 25905 webserver.cc:460] Webserver started at http://127.19.194.62:40517/ using document root <none> and password file <none>
I20260806 18:34:32.541913 25905 fs_manager.cc:359] Metadata directory not provided
I20260806 18:34:32.542012 25905 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 18:34:32.542233 25905 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 18:34:32.544283 25905 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786041246094603-20232-0/minicluster-data/master-0/data/instance:
uuid: "66e3b792019a4cbd8d246a6fd6881e77"
format_stamp: "Formatted at 2026-08-06 18:34:32 on dist-test-slave-1xrh"
server_key: "df3abbf52404780d68464be54f238786"
server_key_iv: "9016393e2e4622309eac9c8a55a1f23d"
server_key_version: "encryptionkey@0"
I20260806 18:34:32.544838 25905 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786041246094603-20232-0/minicluster-data/master-0/wal/instance:
uuid: "66e3b792019a4cbd8d246a6fd6881e77"
format_stamp: "Formatted at 2026-08-06 18:34:32 on dist-test-slave-1xrh"
server_key: "df3abbf52404780d68464be54f238786"
server_key_iv: "9016393e2e4622309eac9c8a55a1f23d"
server_key_version: "encryptionkey@0"
I20260806 18:34:32.548275 25905 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20260806 18:34:32.550586 25919 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 18:34:32.551847 25905 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260806 18:34:32.551990 25905 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786041246094603-20232-0/minicluster-data/master-0/data,/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786041246094603-20232-0/minicluster-data/master-0/wal
uuid: "66e3b792019a4cbd8d246a6fd6881e77"
format_stamp: "Formatted at 2026-08-06 18:34:32 on dist-test-slave-1xrh"
server_key: "df3abbf52404780d68464be54f238786"
server_key_iv: "9016393e2e4622309eac9c8a55a1f23d"
server_key_version: "encryptionkey@0"
I20260806 18:34:32.552099 25905 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786041246094603-20232-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786041246094603-20232-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786041246094603-20232-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 18:34:32.576995 25905 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 18:34:32.577642 25905 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 18:34:32.577838 25905 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 18:34:32.584561 25905 rpc_server.cc:307] RPC server started. Bound to: 127.19.194.62:39547
I20260806 18:34:32.584584 25972 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.194.62:39547 every 8 connection(s)
I20260806 18:34:32.585448 25905 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786041246094603-20232-0/minicluster-data/master-0/data/info.pb
I20260806 18:34:32.586580 20232 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJW69tn/build/debug/bin/kudu as pid 25905
I20260806 18:34:32.586704 20232 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786041246094603-20232-0/minicluster-data/master-0/wal/instance
I20260806 18:34:32.587037 20232 external_mini_cluster.cc:1428] Setting key f51091df0e2e5227426c61cf6509adac
I20260806 18:34:32.588769 25973 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 18:34:32.594235 25973 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 66e3b792019a4cbd8d246a6fd6881e77: Bootstrap starting.
I20260806 18:34:32.595952 25976 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:34:32.588376 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40840 (local address 127.19.194.62:39547)
0806 18:34:32.589062 (+ 686us) server_negotiation.cc:206] Beginning negotiation
0806 18:34:32.589077 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0806 18:34:32.589111 (+ 34us) server_negotiation.cc:407] Connection header received
0806 18:34:32.589748 (+ 637us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:34:32.589763 (+ 15us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:34:32.590134 (+ 371us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:34:32.590469 (+ 335us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:34:32.590889 (+ 420us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:34:32.591860 (+ 971us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:34:32.592741 (+ 881us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:34:32.593187 (+ 446us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:34:32.593351 (+ 164us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:34:32.594220 (+ 869us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:34:32.594225 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:34:32.594237 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 18:34:32.594674 (+ 437us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:34:32.594728 (+ 54us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 18:34:32.594749 (+ 21us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:34:32.594859 (+ 110us) server_negotiation.cc:1035] Waiting for connection context
0806 18:34:32.595009 (+ 150us) server_negotiation.cc:299] Negotiation successful
0806 18:34:32.595279 (+ 270us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":324,"thread_start_us":140,"threads_started":1}
I20260806 18:34:32.596587 25973 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 66e3b792019a4cbd8d246a6fd6881e77: Neither blocks nor log segments found. Creating new log.
I20260806 18:34:32.597443 25973 log.cc:824] T 00000000000000000000000000000000 P 66e3b792019a4cbd8d246a6fd6881e77: Log is configured to *not* fsync() on all Append() calls
I20260806 18:34:32.598775 20232 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260806 18:34:32.599496 25973 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 66e3b792019a4cbd8d246a6fd6881e77: No bootstrap required, opened a new log
I20260806 18:34:32.603049 25973 raft_consensus.cc:348] T 00000000000000000000000000000000 P 66e3b792019a4cbd8d246a6fd6881e77 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "66e3b792019a4cbd8d246a6fd6881e77" member_type: VOTER last_known_addr { host: "127.19.194.62" port: 39547 } }
I20260806 18:34:32.603300 25973 raft_consensus.cc:374] T 00000000000000000000000000000000 P 66e3b792019a4cbd8d246a6fd6881e77 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 18:34:32.603338 25973 raft_consensus.cc:729] T 00000000000000000000000000000000 P 66e3b792019a4cbd8d246a6fd6881e77 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 66e3b792019a4cbd8d246a6fd6881e77, State: Initialized, Role: FOLLOWER
I20260806 18:34:32.603668 25973 consensus_queue.cc:260] T 00000000000000000000000000000000 P 66e3b792019a4cbd8d246a6fd6881e77 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "66e3b792019a4cbd8d246a6fd6881e77" member_type: VOTER last_known_addr { host: "127.19.194.62" port: 39547 } }
I20260806 18:34:32.603775 25973 raft_consensus.cc:388] T 00000000000000000000000000000000 P 66e3b792019a4cbd8d246a6fd6881e77 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 18:34:32.603893 25973 raft_consensus.cc:482] T 00000000000000000000000000000000 P 66e3b792019a4cbd8d246a6fd6881e77 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 18:34:32.603976 25973 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 66e3b792019a4cbd8d246a6fd6881e77 [term 0 FOLLOWER]: Advancing to term 1
I20260806 18:34:32.604758 25973 raft_consensus.cc:504] T 00000000000000000000000000000000 P 66e3b792019a4cbd8d246a6fd6881e77 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "66e3b792019a4cbd8d246a6fd6881e77" member_type: VOTER last_known_addr { host: "127.19.194.62" port: 39547 } }
I20260806 18:34:32.605109 25973 leader_election.cc:302] T 00000000000000000000000000000000 P 66e3b792019a4cbd8d246a6fd6881e77 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 66e3b792019a4cbd8d246a6fd6881e77; no voters:
I20260806 18:34:32.605504 25973 leader_election.cc:288] T 00000000000000000000000000000000 P 66e3b792019a4cbd8d246a6fd6881e77 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 18:34:32.605611 25978 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 66e3b792019a4cbd8d246a6fd6881e77 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 18:34:32.605861 25978 raft_consensus.cc:686] T 00000000000000000000000000000000 P 66e3b792019a4cbd8d246a6fd6881e77 [term 1 LEADER]: Becoming Leader. State: Replica: 66e3b792019a4cbd8d246a6fd6881e77, State: Running, Role: LEADER
I20260806 18:34:32.606184 25978 consensus_queue.cc:237] T 00000000000000000000000000000000 P 66e3b792019a4cbd8d246a6fd6881e77 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "66e3b792019a4cbd8d246a6fd6881e77" member_type: VOTER last_known_addr { host: "127.19.194.62" port: 39547 } }
I20260806 18:34:32.606700 25973 sys_catalog.cc:564] T 00000000000000000000000000000000 P 66e3b792019a4cbd8d246a6fd6881e77 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 18:34:32.607983 25979 sys_catalog.cc:455] T 00000000000000000000000000000000 P 66e3b792019a4cbd8d246a6fd6881e77 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "66e3b792019a4cbd8d246a6fd6881e77" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "66e3b792019a4cbd8d246a6fd6881e77" member_type: VOTER last_known_addr { host: "127.19.194.62" port: 39547 } } }
I20260806 18:34:32.607995 25982 sys_catalog.cc:455] T 00000000000000000000000000000000 P 66e3b792019a4cbd8d246a6fd6881e77 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 66e3b792019a4cbd8d246a6fd6881e77. Latest consensus state: current_term: 1 leader_uuid: "66e3b792019a4cbd8d246a6fd6881e77" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "66e3b792019a4cbd8d246a6fd6881e77" member_type: VOTER last_known_addr { host: "127.19.194.62" port: 39547 } } }
I20260806 18:34:32.608167 25982 sys_catalog.cc:458] T 00000000000000000000000000000000 P 66e3b792019a4cbd8d246a6fd6881e77 [sys.catalog]: This master's current role is: LEADER
I20260806 18:34:32.608166 25979 sys_catalog.cc:458] T 00000000000000000000000000000000 P 66e3b792019a4cbd8d246a6fd6881e77 [sys.catalog]: This master's current role is: LEADER
I20260806 18:34:32.608641 25989 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 18:34:32.611727 25989 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 18:34:32.617113 25989 catalog_manager.cc:1347] Generated new cluster ID: cc433ae164a34b619f0c636137337b21
I20260806 18:34:32.617175 25989 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 18:34:32.623016 25989 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 18:34:32.624145 25989 catalog_manager.cc:1504] Loading token signing keys...
I20260806 18:34:32.632418 25989 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 66e3b792019a4cbd8d246a6fd6881e77: Generated new TSK 0
I20260806 18:34:32.633133 25989 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 18:34:33.113234 25976 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:34:32.641153 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40844 (local address 127.19.194.62:39547)
0806 18:34:32.641292 (+ 139us) server_negotiation.cc:206] Beginning negotiation
0806 18:34:32.641296 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 18:34:32.641513 (+ 217us) server_negotiation.cc:407] Connection header received
0806 18:34:32.641683 (+ 170us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:34:32.641686 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:34:32.641728 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:34:32.641806 (+ 78us) server_negotiation.cc:226] Negotiated authn=JWT
0806 18:34:32.642067 (+ 261us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:34:32.642538 (+ 471us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:34:32.643772 (+ 1234us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:34:32.644018 (+ 246us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:34:32.644096 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:34:32.644432 (+ 336us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0806 18:34:33.112711 (+468279us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
0806 18:34:33.113070 (+ 359us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:40844: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
Metrics: {"server-negotiator.queue_time_us":55}
W20260806 18:34:33.113334 25976 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:40844: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
W20260806 18:34:33.113360 26004 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 18:34:32.641104 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.194.62:39547 (local address 127.0.0.1:40844)
0806 18:34:32.641419 (+ 315us) negotiation.cc:107] Waiting for socket to connect
0806 18:34:32.641434 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0806 18:34:32.641556 (+ 122us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:34:32.641833 (+ 277us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:34:32.641836 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:34:32.641844 (+ 8us) client_negotiation.cc:189] Negotiated authn=JWT
0806 18:34:32.641964 (+ 120us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:34:32.641971 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:34:32.642663 (+ 692us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:34:32.642668 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:34:32.643661 (+ 993us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:34:32.643672 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:34:32.644142 (+ 470us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:34:32.644146 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:34:32.644243 (+ 97us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:34:32.644272 (+ 29us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0806 18:34:33.112956 (+468684us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
0806 18:34:33.113160 (+ 204us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.19.194.62:39547: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
Metrics: {"client-negotiator.queue_time_us":228,"thread_start_us":126,"threads_started":1}
I20260806 18:34:33.113974 20232 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJW69tn/build/debug/bin/kudu with pid 25905
2026-08-06T18:34:33Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS (732 ms)
[ RUN ] SecurityITest.TestWorldReadableKeytab
[ OK ] SecurityITest.TestWorldReadableKeytab (122 ms)
[ RUN ] SecurityITest.TestWorldReadablePrivateKey
[ OK ] SecurityITest.TestWorldReadablePrivateKey (126 ms)
[ RUN ] SecurityITest.TestCorruptKerberosCC
Loading random data
Initializing database '/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786041246094603-20232-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 18:34:33 dist-test-slave-1xrh krb5kdc[26016](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 18:34:33 dist-test-slave-1xrh krb5kdc[26016](info): set up 2 sockets
Aug 06 18:34:33 dist-test-slave-1xrh krb5kdc[26016](info): commencing operation
krb5kdc: starting...
W20260806 18:34:34.421063 20232 mini_kdc.cc:121] Time spent starting KDC: real 1.037s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786041246094603-20232-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786041246094603-20232-0/krb5kdc/test-user.keytab.
Aug 06 18:34:34 dist-test-slave-1xrh krb5kdc[26016](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041274, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T18:34:34Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T18:34:34Z Disabled control of system clock
WARNING: no policy specified for kudu/127.19.194.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.19.194.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.19.194.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786041246094603-20232-0/krb5kdc/kudu_127.19.194.62.keytab.
Entry for principal kudu/127.19.194.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786041246094603-20232-0/krb5kdc/kudu_127.19.194.62.keytab.
I20260806 18:34:34.541885 20232 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJW69tn/build/debug/bin/kudu
/tmp/dist-test-taskJW69tn/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786041246094603-20232-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786041246094603-20232-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786041246094603-20232-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786041246094603-20232-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.19.194.62:37607
--webserver_interface=127.19.194.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786041246094603-20232-0/krb5kdc/kudu_127.19.194.62.keytab
--principal=kudu/127.19.194.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.19.194.20:45305
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.19.194.62:37607
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786041246094603-20232-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786041246094603-20232-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786041246094603-20232-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 18:34:34.649814 26029 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 18:34:34.650095 26029 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 18:34:34.650163 26029 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 18:34:34.654162 26029 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 18:34:34.654248 26029 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 18:34:34.654282 26029 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 18:34:34.654320 26029 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 18:34:34.654354 26029 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 18:34:34.659439 26029 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.194.20:45305
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786041246094603-20232-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786041246094603-20232-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.19.194.62:37607
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786041246094603-20232-0/krb5kdc/kudu_127.19.194.62.keytab
--principal=kudu/127.19.194.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.19.194.62:37607
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786041246094603-20232-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.19.194.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26029
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786041246094603-20232-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 17:59:13 UTC on ccycloud.quasar-wvsnzx.root.comops.site
build id 4571028
I20260806 18:34:34.660427 26029 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 18:34:34.661391 26029 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 18:34:34.669927 26037 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:34:34.669916 26034 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:34:34.669991 26035 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 18:34:34.670166 26029 server_base.cc:1034] running on GCE node
I20260806 18:34:34.670675 26029 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 18:34:34.671831 26029 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 18:34:34.673020 26029 hybrid_clock.cc:648] HybridClock initialized: now 1786041274673002 us; error 31 us; skew 500 ppm
Aug 06 18:34:34 dist-test-slave-1xrh krb5kdc[26016](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041274, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.194.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 18:34:34.675738 26029 init.cc:377] Logged in from keytab as kudu/127.19.194.62@KRBTEST.COM (short username kudu)
I20260806 18:34:34.676508 26029 webserver.cc:460] Webserver started at http://127.19.194.62:35353/ using document root <none> and password file <none>
I20260806 18:34:34.676985 26029 fs_manager.cc:359] Metadata directory not provided
I20260806 18:34:34.677049 26029 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 18:34:34.677209 26029 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 18:34:34.679123 26029 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786041246094603-20232-0/minicluster-data/master-0/data/instance:
uuid: "0e08482c22d543bc9d66b5dd32e413f6"
format_stamp: "Formatted at 2026-08-06 18:34:34 on dist-test-slave-1xrh"
server_key: "90215f4f8e6e556c46f28c2730f8a1e7"
server_key_iv: "24a2804eba94a7faa5c745d6a412a4d4"
server_key_version: "encryptionkey@0"
I20260806 18:34:34.679699 26029 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786041246094603-20232-0/minicluster-data/master-0/wal/instance:
uuid: "0e08482c22d543bc9d66b5dd32e413f6"
format_stamp: "Formatted at 2026-08-06 18:34:34 on dist-test-slave-1xrh"
server_key: "90215f4f8e6e556c46f28c2730f8a1e7"
server_key_iv: "24a2804eba94a7faa5c745d6a412a4d4"
server_key_version: "encryptionkey@0"
I20260806 18:34:34.683373 26029 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.000s
I20260806 18:34:34.685654 26044 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 18:34:34.686805 26029 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260806 18:34:34.686901 26029 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786041246094603-20232-0/minicluster-data/master-0/data,/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786041246094603-20232-0/minicluster-data/master-0/wal
uuid: "0e08482c22d543bc9d66b5dd32e413f6"
format_stamp: "Formatted at 2026-08-06 18:34:34 on dist-test-slave-1xrh"
server_key: "90215f4f8e6e556c46f28c2730f8a1e7"
server_key_iv: "24a2804eba94a7faa5c745d6a412a4d4"
server_key_version: "encryptionkey@0"
I20260806 18:34:34.687001 26029 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786041246094603-20232-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786041246094603-20232-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786041246094603-20232-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 18:34:34.699278 26029 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 18:34:34.701779 26029 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 18:34:34.701920 26029 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 18:34:34.709230 26029 rpc_server.cc:307] RPC server started. Bound to: 127.19.194.62:37607
I20260806 18:34:34.709251 26097 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.194.62:37607 every 8 connection(s)
I20260806 18:34:34.710119 26029 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786041246094603-20232-0/minicluster-data/master-0/data/info.pb
I20260806 18:34:34.712757 26098 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 18:34:34.717454 20232 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJW69tn/build/debug/bin/kudu as pid 26029
I20260806 18:34:34.717592 20232 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786041246094603-20232-0/minicluster-data/master-0/wal/instance
I20260806 18:34:34.717873 20232 external_mini_cluster.cc:1428] Setting key ba0b7565a4447f466cd8a60d1ad28bcd
I20260806 18:34:34.718175 26098 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0e08482c22d543bc9d66b5dd32e413f6: Bootstrap starting.
I20260806 18:34:34.720801 26098 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 0e08482c22d543bc9d66b5dd32e413f6: Neither blocks nor log segments found. Creating new log.
I20260806 18:34:34.721704 26098 log.cc:824] T 00000000000000000000000000000000 P 0e08482c22d543bc9d66b5dd32e413f6: Log is configured to *not* fsync() on all Append() calls
I20260806 18:34:34.723867 26098 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0e08482c22d543bc9d66b5dd32e413f6: No bootstrap required, opened a new log
Aug 06 18:34:34 dist-test-slave-1xrh krb5kdc[26016](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041274, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.19.194.62@KRBTEST.COM
I20260806 18:34:34.727620 26098 raft_consensus.cc:348] T 00000000000000000000000000000000 P 0e08482c22d543bc9d66b5dd32e413f6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0e08482c22d543bc9d66b5dd32e413f6" member_type: VOTER last_known_addr { host: "127.19.194.62" port: 37607 } }
I20260806 18:34:34.727891 26098 raft_consensus.cc:374] T 00000000000000000000000000000000 P 0e08482c22d543bc9d66b5dd32e413f6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 18:34:34.727958 26098 raft_consensus.cc:729] T 00000000000000000000000000000000 P 0e08482c22d543bc9d66b5dd32e413f6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0e08482c22d543bc9d66b5dd32e413f6, State: Initialized, Role: FOLLOWER
I20260806 18:34:34.728509 26098 consensus_queue.cc:260] T 00000000000000000000000000000000 P 0e08482c22d543bc9d66b5dd32e413f6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0e08482c22d543bc9d66b5dd32e413f6" member_type: VOTER last_known_addr { host: "127.19.194.62" port: 37607 } }
I20260806 18:34:34.728695 26098 raft_consensus.cc:388] T 00000000000000000000000000000000 P 0e08482c22d543bc9d66b5dd32e413f6 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 18:34:34.728767 26098 raft_consensus.cc:482] T 00000000000000000000000000000000 P 0e08482c22d543bc9d66b5dd32e413f6 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 18:34:34.728865 26098 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 0e08482c22d543bc9d66b5dd32e413f6 [term 0 FOLLOWER]: Advancing to term 1
I20260806 18:34:34.729701 26098 raft_consensus.cc:504] T 00000000000000000000000000000000 P 0e08482c22d543bc9d66b5dd32e413f6 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0e08482c22d543bc9d66b5dd32e413f6" member_type: VOTER last_known_addr { host: "127.19.194.62" port: 37607 } }
I20260806 18:34:34.730037 26098 leader_election.cc:302] T 00000000000000000000000000000000 P 0e08482c22d543bc9d66b5dd32e413f6 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 0e08482c22d543bc9d66b5dd32e413f6; no voters:
I20260806 18:34:34.730383 26098 leader_election.cc:288] T 00000000000000000000000000000000 P 0e08482c22d543bc9d66b5dd32e413f6 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 18:34:34.730448 26103 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 0e08482c22d543bc9d66b5dd32e413f6 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 18:34:34.730674 26103 raft_consensus.cc:686] T 00000000000000000000000000000000 P 0e08482c22d543bc9d66b5dd32e413f6 [term 1 LEADER]: Becoming Leader. State: Replica: 0e08482c22d543bc9d66b5dd32e413f6, State: Running, Role: LEADER
I20260806 18:34:34.731024 26103 consensus_queue.cc:237] T 00000000000000000000000000000000 P 0e08482c22d543bc9d66b5dd32e413f6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0e08482c22d543bc9d66b5dd32e413f6" member_type: VOTER last_known_addr { host: "127.19.194.62" port: 37607 } }
I20260806 18:34:34.731652 26098 sys_catalog.cc:564] T 00000000000000000000000000000000 P 0e08482c22d543bc9d66b5dd32e413f6 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 18:34:34.732956 26101 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:34:34.719246 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54042 (local address 127.19.194.62:37607)
0806 18:34:34.719647 (+ 401us) server_negotiation.cc:206] Beginning negotiation
0806 18:34:34.719665 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0806 18:34:34.719706 (+ 41us) server_negotiation.cc:407] Connection header received
0806 18:34:34.720507 (+ 801us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:34:34.720529 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:34:34.720936 (+ 407us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:34:34.721323 (+ 387us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:34:34.722063 (+ 740us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:34:34.723427 (+ 1364us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:34:34.724352 (+ 925us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:34:34.725028 (+ 676us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:34:34.725203 (+ 175us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:34:34.727663 (+ 2460us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:34:34.727670 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:34:34.727685 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 18:34:34.727715 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:34:34.729569 (+ 1854us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:34:34.730087 (+ 518us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:34:34.730099 (+ 12us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:34:34.730103 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:34:34.730264 (+ 161us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:34:34.730622 (+ 358us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:34:34.730653 (+ 31us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:34:34.730656 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:34:34.731225 (+ 569us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:34:34.731447 (+ 222us) server_negotiation.cc:1035] Waiting for connection context
0806 18:34:34.731738 (+ 291us) server_negotiation.cc:299] Negotiation successful
0806 18:34:34.732036 (+ 298us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":256,"thread_start_us":95,"threads_started":1}
I20260806 18:34:34.732986 26105 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0e08482c22d543bc9d66b5dd32e413f6 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 0e08482c22d543bc9d66b5dd32e413f6. Latest consensus state: current_term: 1 leader_uuid: "0e08482c22d543bc9d66b5dd32e413f6" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0e08482c22d543bc9d66b5dd32e413f6" member_type: VOTER last_known_addr { host: "127.19.194.62" port: 37607 } } }
I20260806 18:34:34.733106 26105 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0e08482c22d543bc9d66b5dd32e413f6 [sys.catalog]: This master's current role is: LEADER
I20260806 18:34:34.732991 26104 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0e08482c22d543bc9d66b5dd32e413f6 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "0e08482c22d543bc9d66b5dd32e413f6" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0e08482c22d543bc9d66b5dd32e413f6" member_type: VOTER last_known_addr { host: "127.19.194.62" port: 37607 } } }
I20260806 18:34:34.733388 26104 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0e08482c22d543bc9d66b5dd32e413f6 [sys.catalog]: This master's current role is: LEADER
W20260806 18:34:34.736996 26119 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 0e08482c22d543bc9d66b5dd32e413f6: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260806 18:34:34.737089 26119 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260806 18:34:34.737171 26109 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 18:34:34.737854 26109 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 18:34:34.742878 26109 catalog_manager.cc:1347] Generated new cluster ID: 8776dd5d88194ec6b272dcf5a1cd2a82
I20260806 18:34:34.742949 26109 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 18:34:34.758096 26109 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 18:34:34.759174 26109 catalog_manager.cc:1504] Loading token signing keys...
I20260806 18:34:34.764683 26109 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 0e08482c22d543bc9d66b5dd32e413f6: Generated new TSK 0
I20260806 18:34:34.765328 26109 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.19.194.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.19.194.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.19.194.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786041246094603-20232-0/krb5kdc/kudu_127.19.194.1.keytab.
Entry for principal kudu/127.19.194.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786041246094603-20232-0/krb5kdc/kudu_127.19.194.1.keytab.
I20260806 18:34:34.795035 20232 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJW69tn/build/debug/bin/kudu
/tmp/dist-test-taskJW69tn/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786041246094603-20232-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786041246094603-20232-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786041246094603-20232-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786041246094603-20232-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.19.194.1:0
--local_ip_for_outbound_sockets=127.19.194.1
--webserver_interface=127.19.194.1
--webserver_port=0
--tserver_master_addrs=127.19.194.62:37607
--keytab_file=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786041246094603-20232-0/krb5kdc/kudu_127.19.194.1.keytab
--principal=kudu/127.19.194.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.19.194.20:45305
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786041246094603-20232-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786041246094603-20232-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786041246094603-20232-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 18:34:34.914906 26125 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 18:34:34.915261 26125 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 18:34:34.915366 26125 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 18:34:34.920037 26125 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 18:34:34.920114 26125 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 18:34:34.920214 26125 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.19.194.1
I20260806 18:34:34.926028 26125 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.194.20:45305
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786041246094603-20232-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786041246094603-20232-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786041246094603-20232-0/krb5kdc/kudu_127.19.194.1.keytab
--principal=kudu/127.19.194.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.19.194.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786041246094603-20232-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.19.194.1
--webserver_port=0
--tserver_master_addrs=127.19.194.62:37607
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26125
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.19.194.1
--log_dir=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786041246094603-20232-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 17:59:13 UTC on ccycloud.quasar-wvsnzx.root.comops.site
build id 4571028
I20260806 18:34:34.927385 26125 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 18:34:34.928648 26125 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 18:34:34.940129 26130 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:34:34.940496 26133 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:34:34.941301 26131 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 18:34:34.941679 26125 server_base.cc:1034] running on GCE node
I20260806 18:34:34.942173 26125 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 18:34:34.942878 26125 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 18:34:34.944064 26125 hybrid_clock.cc:648] HybridClock initialized: now 1786041274944046 us; error 29 us; skew 500 ppm
Aug 06 18:34:34 dist-test-slave-1xrh krb5kdc[26016](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041274, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.194.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 18:34:34.946733 26125 init.cc:377] Logged in from keytab as kudu/127.19.194.1@KRBTEST.COM (short username kudu)
I20260806 18:34:34.947579 26125 webserver.cc:460] Webserver started at http://127.19.194.1:40885/ using document root <none> and password file <none>
I20260806 18:34:34.948076 26125 fs_manager.cc:359] Metadata directory not provided
I20260806 18:34:34.948130 26125 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 18:34:34.948282 26125 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 18:34:34.950291 26125 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786041246094603-20232-0/minicluster-data/ts-0/data/instance:
uuid: "d887c671ce5845a4b6899c7e11e08a86"
format_stamp: "Formatted at 2026-08-06 18:34:34 on dist-test-slave-1xrh"
server_key: "0d70bef98004571c025ba47e31937c64"
server_key_iv: "ddffd5785480b8c9e6ba2980bebdb904"
server_key_version: "encryptionkey@0"
I20260806 18:34:34.950801 26125 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786041246094603-20232-0/minicluster-data/ts-0/wal/instance:
uuid: "d887c671ce5845a4b6899c7e11e08a86"
format_stamp: "Formatted at 2026-08-06 18:34:34 on dist-test-slave-1xrh"
server_key: "0d70bef98004571c025ba47e31937c64"
server_key_iv: "ddffd5785480b8c9e6ba2980bebdb904"
server_key_version: "encryptionkey@0"
I20260806 18:34:34.954690 26125 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260806 18:34:34.957203 26140 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 18:34:34.958424 26125 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260806 18:34:34.958520 26125 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786041246094603-20232-0/minicluster-data/ts-0/data,/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786041246094603-20232-0/minicluster-data/ts-0/wal
uuid: "d887c671ce5845a4b6899c7e11e08a86"
format_stamp: "Formatted at 2026-08-06 18:34:34 on dist-test-slave-1xrh"
server_key: "0d70bef98004571c025ba47e31937c64"
server_key_iv: "ddffd5785480b8c9e6ba2980bebdb904"
server_key_version: "encryptionkey@0"
I20260806 18:34:34.958606 26125 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786041246094603-20232-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786041246094603-20232-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786041246094603-20232-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 18:34:34.973456 26125 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 18:34:34.976284 26125 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 18:34:34.976439 26125 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 18:34:34.976963 26125 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 18:34:34.977756 26125 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 18:34:34.977805 26125 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 18:34:34.977845 26125 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 18:34:34.977860 26125 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 18:34:34.986706 26125 rpc_server.cc:307] RPC server started. Bound to: 127.19.194.1:35847
I20260806 18:34:34.986765 26256 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.194.1:35847 every 8 connection(s)
I20260806 18:34:34.987643 26125 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786041246094603-20232-0/minicluster-data/ts-0/data/info.pb
I20260806 18:34:34.991734 20232 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJW69tn/build/debug/bin/kudu as pid 26125
I20260806 18:34:34.991864 20232 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786041246094603-20232-0/minicluster-data/ts-0/wal/instance
I20260806 18:34:34.992199 20232 external_mini_cluster.cc:1428] Setting key 275a94d3aa2e7d3628718e541bb9564e
Aug 06 18:34:34 dist-test-slave-1xrh krb5kdc[26016](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041274, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.194.1@KRBTEST.COM for kudu/127.19.194.62@KRBTEST.COM
I20260806 18:34:35.003779 26101 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:34:34.989467 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.194.1:58277 (local address 127.19.194.62:37607)
0806 18:34:34.989614 (+ 147us) server_negotiation.cc:206] Beginning negotiation
0806 18:34:34.989619 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 18:34:34.990465 (+ 846us) server_negotiation.cc:407] Connection header received
0806 18:34:34.992189 (+ 1724us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:34:34.992199 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:34:34.992271 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:34:34.992356 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:34:34.993863 (+ 1507us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:34:34.994427 (+ 564us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:34:34.996148 (+ 1721us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:34:34.996546 (+ 398us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:34:34.996679 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:34:34.999693 (+ 3014us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:34:34.999700 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:34:34.999706 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 18:34:34.999743 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:34:35.001502 (+ 1759us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:34:35.002036 (+ 534us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:34:35.002043 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:34:35.002045 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:34:35.002147 (+ 102us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:34:35.002487 (+ 340us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:34:35.002494 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:34:35.002499 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:34:35.002766 (+ 267us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:34:35.002872 (+ 106us) server_negotiation.cc:1035] Waiting for connection context
0806 18:34:35.003438 (+ 566us) server_negotiation.cc:299] Negotiation successful
0806 18:34:35.003599 (+ 161us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
WARNING: no policy specified for kudu/127.19.194.2@KRBTEST.COM; defaulting to no policy
I20260806 18:34:35.004576 26260 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:34:34.989890 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.194.62:37607 (local address 127.19.194.1:58277)
0806 18:34:34.990328 (+ 438us) negotiation.cc:107] Waiting for socket to connect
0806 18:34:34.990371 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0806 18:34:34.991941 (+ 1570us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:34:34.992579 (+ 638us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:34:34.992590 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:34:34.993092 (+ 502us) client_negotiation.cc:189] Negotiated authn=SASL
0806 18:34:34.993700 (+ 608us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:34:34.993745 (+ 45us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:34:34.994569 (+ 824us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:34:34.994576 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:34:34.996008 (+ 1432us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:34:34.996019 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:34:34.996883 (+ 864us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:34:34.996889 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:34:34.997143 (+ 254us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:34:34.997755 (+ 612us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 18:34:34.997780 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0806 18:34:34.999501 (+ 1721us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 18:34:35.001632 (+ 2131us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:34:35.001636 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:34:35.001650 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:34:35.001897 (+ 247us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:34:35.002247 (+ 350us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:34:35.002249 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:34:35.002250 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:34:35.002378 (+ 128us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:34:35.002872 (+ 494us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 18:34:35.002877 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 18:34:35.003109 (+ 232us) client_negotiation.cc:769] Sending connection context
0806 18:34:35.003325 (+ 216us) client_negotiation.cc:240] Negotiation successful
0806 18:34:35.003610 (+ 285us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":240,"thread_start_us":105,"threads_started":1}
I20260806 18:34:35.005884 26258 heartbeater.cc:347] Connected to a master server at 127.19.194.62:37607
I20260806 18:34:35.006134 26258 heartbeater.cc:464] Registering TS with master...
I20260806 18:34:35.006667 26258 heartbeater.cc:511] Master 127.19.194.62:37607 requested a full tablet report, sending...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.19.194.2@KRBTEST.COM" created.
I20260806 18:34:35.008314 26061 ts_manager.cc:194] Registered new tserver with Master: d887c671ce5845a4b6899c7e11e08a86 (127.19.194.1:35847)
I20260806 18:34:35.009496 26061 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.19.194.1@KRBTEST.COM'} at 127.19.194.1:58277
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.19.194.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786041246094603-20232-0/krb5kdc/kudu_127.19.194.2.keytab.
Entry for principal kudu/127.19.194.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786041246094603-20232-0/krb5kdc/kudu_127.19.194.2.keytab.
I20260806 18:34:35.022434 20232 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJW69tn/build/debug/bin/kudu
/tmp/dist-test-taskJW69tn/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786041246094603-20232-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786041246094603-20232-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786041246094603-20232-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786041246094603-20232-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.19.194.2:0
--local_ip_for_outbound_sockets=127.19.194.2
--webserver_interface=127.19.194.2
--webserver_port=0
--tserver_master_addrs=127.19.194.62:37607
--keytab_file=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786041246094603-20232-0/krb5kdc/kudu_127.19.194.2.keytab
--principal=kudu/127.19.194.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.19.194.20:45305
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786041246094603-20232-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786041246094603-20232-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786041246094603-20232-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 18:34:35.132294 26263 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 18:34:35.132544 26263 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 18:34:35.132617 26263 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 18:34:35.137152 26263 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 18:34:35.137207 26263 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 18:34:35.137318 26263 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.19.194.2
I20260806 18:34:35.142627 26263 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.194.20:45305
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786041246094603-20232-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786041246094603-20232-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786041246094603-20232-0/krb5kdc/kudu_127.19.194.2.keytab
--principal=kudu/127.19.194.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.19.194.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786041246094603-20232-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.19.194.2
--webserver_port=0
--tserver_master_addrs=127.19.194.62:37607
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26263
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.19.194.2
--log_dir=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786041246094603-20232-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 17:59:13 UTC on ccycloud.quasar-wvsnzx.root.comops.site
build id 4571028
I20260806 18:34:35.143638 26263 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 18:34:35.144641 26263 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 18:34:35.154000 26269 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:34:35.154013 26268 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 18:34:35.154119 26263 server_base.cc:1034] running on GCE node
W20260806 18:34:35.154013 26271 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 18:34:35.154801 26263 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 18:34:35.155483 26263 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 18:34:35.156653 26263 hybrid_clock.cc:648] HybridClock initialized: now 1786041275156622 us; error 44 us; skew 500 ppm
Aug 06 18:34:35 dist-test-slave-1xrh krb5kdc[26016](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041275, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.194.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 18:34:35.159438 26263 init.cc:377] Logged in from keytab as kudu/127.19.194.2@KRBTEST.COM (short username kudu)
I20260806 18:34:35.160295 26263 webserver.cc:460] Webserver started at http://127.19.194.2:44029/ using document root <none> and password file <none>
I20260806 18:34:35.160910 26263 fs_manager.cc:359] Metadata directory not provided
I20260806 18:34:35.160979 26263 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 18:34:35.161259 26263 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 18:34:35.163544 26263 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786041246094603-20232-0/minicluster-data/ts-1/data/instance:
uuid: "32cfb2d100d14294a5d6904156075965"
format_stamp: "Formatted at 2026-08-06 18:34:35 on dist-test-slave-1xrh"
server_key: "0f1318d9f658319311368410f2b51c86"
server_key_iv: "034d4c34b106f36175e0120857c8c4de"
server_key_version: "encryptionkey@0"
I20260806 18:34:35.164105 26263 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786041246094603-20232-0/minicluster-data/ts-1/wal/instance:
uuid: "32cfb2d100d14294a5d6904156075965"
format_stamp: "Formatted at 2026-08-06 18:34:35 on dist-test-slave-1xrh"
server_key: "0f1318d9f658319311368410f2b51c86"
server_key_iv: "034d4c34b106f36175e0120857c8c4de"
server_key_version: "encryptionkey@0"
I20260806 18:34:35.167623 26263 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.000s
I20260806 18:34:35.170054 26278 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 18:34:35.171252 26263 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260806 18:34:35.171342 26263 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786041246094603-20232-0/minicluster-data/ts-1/data,/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786041246094603-20232-0/minicluster-data/ts-1/wal
uuid: "32cfb2d100d14294a5d6904156075965"
format_stamp: "Formatted at 2026-08-06 18:34:35 on dist-test-slave-1xrh"
server_key: "0f1318d9f658319311368410f2b51c86"
server_key_iv: "034d4c34b106f36175e0120857c8c4de"
server_key_version: "encryptionkey@0"
I20260806 18:34:35.171419 26263 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786041246094603-20232-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786041246094603-20232-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786041246094603-20232-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 18:34:35.190270 26263 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 18:34:35.193550 26263 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 18:34:35.193753 26263 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 18:34:35.194321 26263 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 18:34:35.195183 26263 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 18:34:35.195235 26263 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 18:34:35.195297 26263 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 18:34:35.195333 26263 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 18:34:35.205065 26263 rpc_server.cc:307] RPC server started. Bound to: 127.19.194.2:45209
I20260806 18:34:35.205129 26392 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.194.2:45209 every 8 connection(s)
I20260806 18:34:35.205996 26263 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786041246094603-20232-0/minicluster-data/ts-1/data/info.pb
I20260806 18:34:35.208837 20232 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJW69tn/build/debug/bin/kudu as pid 26263
I20260806 18:34:35.208957 20232 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786041246094603-20232-0/minicluster-data/ts-1/wal/instance
I20260806 18:34:35.209250 20232 external_mini_cluster.cc:1428] Setting key 253932f3dc721bb93b1cae3ad89f36ac
Aug 06 18:34:35 dist-test-slave-1xrh krb5kdc[26016](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041275, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.194.2@KRBTEST.COM for kudu/127.19.194.62@KRBTEST.COM
WARNING: no policy specified for kudu/127.19.194.3@KRBTEST.COM; defaulting to no policy
I20260806 18:34:35.220916 26101 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:34:35.207834 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.194.2:40417 (local address 127.19.194.62:37607)
0806 18:34:35.207975 (+ 141us) server_negotiation.cc:206] Beginning negotiation
0806 18:34:35.207979 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 18:34:35.208790 (+ 811us) server_negotiation.cc:407] Connection header received
0806 18:34:35.210367 (+ 1577us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:34:35.210371 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:34:35.210426 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:34:35.210543 (+ 117us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:34:35.212117 (+ 1574us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:34:35.212702 (+ 585us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:34:35.213881 (+ 1179us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:34:35.214110 (+ 229us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:34:35.214209 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:34:35.217102 (+ 2893us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:34:35.217105 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:34:35.217107 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 18:34:35.217130 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:34:35.218610 (+ 1480us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:34:35.219276 (+ 666us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:34:35.219278 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:34:35.219280 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:34:35.219355 (+ 75us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:34:35.219797 (+ 442us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:34:35.219799 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:34:35.219800 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:34:35.219980 (+ 180us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:34:35.220104 (+ 124us) server_negotiation.cc:1035] Waiting for connection context
0806 18:34:35.220679 (+ 575us) server_negotiation.cc:299] Negotiation successful
0806 18:34:35.220789 (+ 110us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260806 18:34:35.221742 26395 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:34:35.208206 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.194.62:37607 (local address 127.19.194.2:40417)
0806 18:34:35.208648 (+ 442us) negotiation.cc:107] Waiting for socket to connect
0806 18:34:35.208691 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0806 18:34:35.210163 (+ 1472us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:34:35.210741 (+ 578us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:34:35.210750 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:34:35.211275 (+ 525us) client_negotiation.cc:189] Negotiated authn=SASL
0806 18:34:35.211946 (+ 671us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:34:35.211977 (+ 31us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:34:35.212859 (+ 882us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:34:35.212863 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:34:35.213750 (+ 887us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:34:35.213759 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:34:35.214260 (+ 501us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:34:35.214262 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:34:35.214405 (+ 143us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:34:35.215002 (+ 597us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 18:34:35.215018 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0806 18:34:35.216928 (+ 1910us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 18:34:35.218736 (+ 1808us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:34:35.218743 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:34:35.218758 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:34:35.219168 (+ 410us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:34:35.219463 (+ 295us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:34:35.219468 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:34:35.219473 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:34:35.219690 (+ 217us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:34:35.220127 (+ 437us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 18:34:35.220133 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 18:34:35.220444 (+ 311us) client_negotiation.cc:769] Sending connection context
0806 18:34:35.220608 (+ 164us) client_negotiation.cc:240] Negotiation successful
0806 18:34:35.220860 (+ 252us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":248,"thread_start_us":109,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.19.194.3@KRBTEST.COM" created.
I20260806 18:34:35.223158 26393 heartbeater.cc:347] Connected to a master server at 127.19.194.62:37607
I20260806 18:34:35.223469 26393 heartbeater.cc:464] Registering TS with master...
I20260806 18:34:35.224046 26393 heartbeater.cc:511] Master 127.19.194.62:37607 requested a full tablet report, sending...
I20260806 18:34:35.225144 26061 ts_manager.cc:194] Registered new tserver with Master: 32cfb2d100d14294a5d6904156075965 (127.19.194.2:45209)
I20260806 18:34:35.225740 26061 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.19.194.2@KRBTEST.COM'} at 127.19.194.2:40417
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.19.194.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786041246094603-20232-0/krb5kdc/kudu_127.19.194.3.keytab.
Entry for principal kudu/127.19.194.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786041246094603-20232-0/krb5kdc/kudu_127.19.194.3.keytab.
I20260806 18:34:35.240725 20232 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJW69tn/build/debug/bin/kudu
/tmp/dist-test-taskJW69tn/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786041246094603-20232-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786041246094603-20232-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786041246094603-20232-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786041246094603-20232-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.19.194.3:0
--local_ip_for_outbound_sockets=127.19.194.3
--webserver_interface=127.19.194.3
--webserver_port=0
--tserver_master_addrs=127.19.194.62:37607
--keytab_file=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786041246094603-20232-0/krb5kdc/kudu_127.19.194.3.keytab
--principal=kudu/127.19.194.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.19.194.20:45305
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786041246094603-20232-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786041246094603-20232-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786041246094603-20232-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 18:34:35.360384 26398 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 18:34:35.360682 26398 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 18:34:35.360725 26398 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 18:34:35.365353 26398 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 18:34:35.365417 26398 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 18:34:35.365478 26398 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.19.194.3
I20260806 18:34:35.371397 26398 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.194.20:45305
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786041246094603-20232-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786041246094603-20232-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786041246094603-20232-0/krb5kdc/kudu_127.19.194.3.keytab
--principal=kudu/127.19.194.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.19.194.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786041246094603-20232-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.19.194.3
--webserver_port=0
--tserver_master_addrs=127.19.194.62:37607
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26398
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.19.194.3
--log_dir=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786041246094603-20232-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 17:59:13 UTC on ccycloud.quasar-wvsnzx.root.comops.site
build id 4571028
I20260806 18:34:35.372409 26398 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 18:34:35.373365 26398 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 18:34:35.382954 26406 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 18:34:35.383056 26398 server_base.cc:1034] running on GCE node
W20260806 18:34:35.383107 26403 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:34:35.383215 26404 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 18:34:35.383757 26398 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 18:34:35.384387 26398 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 18:34:35.385592 26398 hybrid_clock.cc:648] HybridClock initialized: now 1786041275385562 us; error 44 us; skew 500 ppm
Aug 06 18:34:35 dist-test-slave-1xrh krb5kdc[26016](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041275, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.194.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 18:34:35.388554 26398 init.cc:377] Logged in from keytab as kudu/127.19.194.3@KRBTEST.COM (short username kudu)
I20260806 18:34:35.389400 26398 webserver.cc:460] Webserver started at http://127.19.194.3:33315/ using document root <none> and password file <none>
I20260806 18:34:35.389976 26398 fs_manager.cc:359] Metadata directory not provided
I20260806 18:34:35.390076 26398 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 18:34:35.390311 26398 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 18:34:35.392378 26398 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786041246094603-20232-0/minicluster-data/ts-2/data/instance:
uuid: "105ec07c516948aea3d9082234429ea8"
format_stamp: "Formatted at 2026-08-06 18:34:35 on dist-test-slave-1xrh"
server_key: "d4547a395420d0a211d10fb298652f59"
server_key_iv: "ec55fba60a8ec6ccce4798b25c1997cb"
server_key_version: "encryptionkey@0"
I20260806 18:34:35.393014 26398 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786041246094603-20232-0/minicluster-data/ts-2/wal/instance:
uuid: "105ec07c516948aea3d9082234429ea8"
format_stamp: "Formatted at 2026-08-06 18:34:35 on dist-test-slave-1xrh"
server_key: "d4547a395420d0a211d10fb298652f59"
server_key_iv: "ec55fba60a8ec6ccce4798b25c1997cb"
server_key_version: "encryptionkey@0"
I20260806 18:34:35.396719 26398 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.001s sys 0.004s
I20260806 18:34:35.399017 26413 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 18:34:35.400321 26398 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260806 18:34:35.400501 26398 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786041246094603-20232-0/minicluster-data/ts-2/data,/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786041246094603-20232-0/minicluster-data/ts-2/wal
uuid: "105ec07c516948aea3d9082234429ea8"
format_stamp: "Formatted at 2026-08-06 18:34:35 on dist-test-slave-1xrh"
server_key: "d4547a395420d0a211d10fb298652f59"
server_key_iv: "ec55fba60a8ec6ccce4798b25c1997cb"
server_key_version: "encryptionkey@0"
I20260806 18:34:35.400620 26398 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786041246094603-20232-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786041246094603-20232-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786041246094603-20232-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 18:34:35.414345 26398 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 18:34:35.417662 26398 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 18:34:35.417874 26398 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 18:34:35.418462 26398 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 18:34:35.419384 26398 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 18:34:35.419457 26398 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 18:34:35.419529 26398 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 18:34:35.419572 26398 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 18:34:35.428201 26398 rpc_server.cc:307] RPC server started. Bound to: 127.19.194.3:45321
I20260806 18:34:35.428277 26528 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.194.3:45321 every 8 connection(s)
I20260806 18:34:35.429128 26398 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786041246094603-20232-0/minicluster-data/ts-2/data/info.pb
I20260806 18:34:35.437350 20232 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJW69tn/build/debug/bin/kudu as pid 26398
I20260806 18:34:35.437518 20232 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786041246094603-20232-0/minicluster-data/ts-2/wal/instance
I20260806 18:34:35.437865 20232 external_mini_cluster.cc:1428] Setting key fe7e50137e0afa883bfb2598b24f0573
Aug 06 18:34:35 dist-test-slave-1xrh krb5kdc[26016](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041275, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.194.3@KRBTEST.COM for kudu/127.19.194.62@KRBTEST.COM
I20260806 18:34:35.442967 26101 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:34:35.430949 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.194.3:60417 (local address 127.19.194.62:37607)
0806 18:34:35.431101 (+ 152us) server_negotiation.cc:206] Beginning negotiation
0806 18:34:35.431104 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 18:34:35.432007 (+ 903us) server_negotiation.cc:407] Connection header received
0806 18:34:35.433480 (+ 1473us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:34:35.433483 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:34:35.433534 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:34:35.433609 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:34:35.434944 (+ 1335us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:34:35.435511 (+ 567us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:34:35.436631 (+ 1120us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:34:35.436883 (+ 252us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:34:35.436979 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:34:35.439798 (+ 2819us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:34:35.439801 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:34:35.439803 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 18:34:35.439826 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:34:35.441203 (+ 1377us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:34:35.441671 (+ 468us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:34:35.441674 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:34:35.441675 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:34:35.441745 (+ 70us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:34:35.442074 (+ 329us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:34:35.442076 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:34:35.442077 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:34:35.442241 (+ 164us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:34:35.442314 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0806 18:34:35.442759 (+ 445us) server_negotiation.cc:299] Negotiation successful
0806 18:34:35.442848 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260806 18:34:35.443873 26531 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:34:35.431316 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.194.62:37607 (local address 127.19.194.3:60417)
0806 18:34:35.431849 (+ 533us) negotiation.cc:107] Waiting for socket to connect
0806 18:34:35.431892 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0806 18:34:35.433283 (+ 1391us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:34:35.433744 (+ 461us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:34:35.433751 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:34:35.434318 (+ 567us) client_negotiation.cc:189] Negotiated authn=SASL
0806 18:34:35.434789 (+ 471us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:34:35.434803 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:34:35.435631 (+ 828us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:34:35.435633 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:34:35.436499 (+ 866us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:34:35.436508 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:34:35.437034 (+ 526us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:34:35.437037 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:34:35.437168 (+ 131us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:34:35.437793 (+ 625us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 18:34:35.437808 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0806 18:34:35.439625 (+ 1817us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 18:34:35.441307 (+ 1682us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:34:35.441312 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:34:35.441323 (+ 11us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:34:35.441570 (+ 247us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:34:35.441843 (+ 273us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:34:35.441845 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:34:35.441846 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:34:35.441993 (+ 147us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:34:35.442347 (+ 354us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 18:34:35.442352 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 18:34:35.442563 (+ 211us) client_negotiation.cc:769] Sending connection context
0806 18:34:35.442714 (+ 151us) client_negotiation.cc:240] Negotiation successful
0806 18:34:35.442988 (+ 274us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":338,"thread_start_us":124,"threads_started":1}
I20260806 18:34:35.444983 26529 heartbeater.cc:347] Connected to a master server at 127.19.194.62:37607
I20260806 18:34:35.445185 26529 heartbeater.cc:464] Registering TS with master...
I20260806 18:34:35.445588 26529 heartbeater.cc:511] Master 127.19.194.62:37607 requested a full tablet report, sending...
I20260806 18:34:35.446529 26061 ts_manager.cc:194] Registered new tserver with Master: 105ec07c516948aea3d9082234429ea8 (127.19.194.3:45321)
I20260806 18:34:35.447083 26061 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.19.194.3@KRBTEST.COM'} at 127.19.194.3:60417
I20260806 18:34:35.451854 20232 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal test-admin with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786041246094603-20232-0/krb5kdc/test-admin.keytab.
Entry for principal test-admin with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786041246094603-20232-0/krb5kdc/test-admin.keytab.
Aug 06 18:34:35 dist-test-slave-1xrh krb5kdc[26016](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041275, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 18:34:35.468436 20232 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
W20260806 18:34:35.472328 20232 init.cc:244] Error while opening credential cache '/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786041246094603-20232-0/krb5kdc/krb5cc' of type FILE: Runtime error: Bad format in credentials cache (filename: /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786041246094603-20232-0/krb5kdc/krb5cc)
W20260806 18:34:35.472532 26536 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 18:34:35.470292 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57562 (local address 127.19.194.1:35847)
0806 18:34:35.470667 (+ 375us) server_negotiation.cc:206] Beginning negotiation
0806 18:34:35.470686 (+ 19us) server_negotiation.cc:399] Waiting for connection header
0806 18:34:35.470720 (+ 34us) server_negotiation.cc:407] Connection header received
0806 18:34:35.470788 (+ 68us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:34:35.470797 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:34:35.471070 (+ 273us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:34:35.471273 (+ 203us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:34:35.472348 (+ 1075us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:57562: BlockingRecv error: recv got EOF from 127.0.0.1:57562 (error 108)
Metrics: {"server-negotiator.queue_time_us":212,"thread_start_us":103,"threads_started":1}
Aug 06 18:34:35 dist-test-slave-1xrh krb5kdc[26016](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041275, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 18:34:35.473580 20232 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Aug 06 18:34:35 dist-test-slave-1xrh krb5kdc[26016](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041275, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 18:34:35.474604 20232 init.cc:295] Successfully reacquired a new kerberos TGT
Aug 06 18:34:35 dist-test-slave-1xrh krb5kdc[26016](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041275, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.19.194.1@KRBTEST.COM
I20260806 18:34:35.484086 26536 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:34:35.475626 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57568 (local address 127.19.194.1:35847)
0806 18:34:35.475778 (+ 152us) server_negotiation.cc:206] Beginning negotiation
0806 18:34:35.475783 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 18:34:35.475797 (+ 14us) server_negotiation.cc:407] Connection header received
0806 18:34:35.475904 (+ 107us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:34:35.475906 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:34:35.475944 (+ 38us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:34:35.476011 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:34:35.476514 (+ 503us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:34:35.477285 (+ 771us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:34:35.478095 (+ 810us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:34:35.478449 (+ 354us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:34:35.478543 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:34:35.480805 (+ 2262us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:34:35.480815 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:34:35.480828 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 18:34:35.480850 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:34:35.482287 (+ 1437us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:34:35.482749 (+ 462us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:34:35.482754 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:34:35.482755 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:34:35.482832 (+ 77us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:34:35.483142 (+ 310us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:34:35.483146 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:34:35.483147 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:34:35.483383 (+ 236us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:34:35.483610 (+ 227us) server_negotiation.cc:1035] Waiting for connection context
0806 18:34:35.483798 (+ 188us) server_negotiation.cc:299] Negotiation successful
0806 18:34:35.483952 (+ 154us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
W20260806 18:34:35.487561 26536 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 18:34:35.486638 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57570 (local address 127.19.194.1:35847)
0806 18:34:35.486756 (+ 118us) server_negotiation.cc:206] Beginning negotiation
0806 18:34:35.486762 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 18:34:35.486879 (+ 117us) server_negotiation.cc:407] Connection header received
0806 18:34:35.486956 (+ 77us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:34:35.486958 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:34:35.486993 (+ 35us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:34:35.487053 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:34:35.487484 (+ 431us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:57570: BlockingRecv error: recv got EOF from 127.0.0.1:57570 (error 108)
Metrics: {"server-negotiator.queue_time_us":48}
W20260806 18:34:35.487669 20232 init.cc:300] Could not find kerberos principal in credential cache '/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786041246094603-20232-0/krb5kdc/krb5cc' of type FILE
Aug 06 18:34:35 dist-test-slave-1xrh krb5kdc[26016](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041275, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 18:34:35.488647 20232 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Aug 06 18:34:35 dist-test-slave-1xrh krb5kdc[26016](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041275, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 18:34:35.489495 20232 init.cc:295] Successfully reacquired a new kerberos TGT
Aug 06 18:34:35 dist-test-slave-1xrh krb5kdc[26016](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041275, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.19.194.1@KRBTEST.COM
I20260806 18:34:35.497653 26536 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:34:35.490338 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57572 (local address 127.19.194.1:35847)
0806 18:34:35.490472 (+ 134us) server_negotiation.cc:206] Beginning negotiation
0806 18:34:35.490475 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 18:34:35.490491 (+ 16us) server_negotiation.cc:407] Connection header received
0806 18:34:35.490596 (+ 105us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:34:35.490597 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:34:35.490629 (+ 32us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:34:35.490692 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:34:35.491309 (+ 617us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:34:35.491799 (+ 490us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:34:35.492506 (+ 707us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:34:35.492711 (+ 205us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:34:35.492785 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:34:35.494853 (+ 2068us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:34:35.494856 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:34:35.494857 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 18:34:35.494878 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:34:35.496293 (+ 1415us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:34:35.496658 (+ 365us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:34:35.496660 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:34:35.496662 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:34:35.496729 (+ 67us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:34:35.496949 (+ 220us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:34:35.496951 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:34:35.496952 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:34:35.497100 (+ 148us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:34:35.497189 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0806 18:34:35.497368 (+ 179us) server_negotiation.cc:299] Negotiation successful
0806 18:34:35.497483 (+ 115us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
W20260806 18:34:35.500092 26536 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 18:34:35.499247 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57580 (local address 127.19.194.1:35847)
0806 18:34:35.499369 (+ 122us) server_negotiation.cc:206] Beginning negotiation
0806 18:34:35.499373 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 18:34:35.499413 (+ 40us) server_negotiation.cc:407] Connection header received
0806 18:34:35.499508 (+ 95us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:34:35.499510 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:34:35.499540 (+ 30us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:34:35.499598 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:34:35.499997 (+ 399us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:57580: BlockingRecv error: recv got EOF from 127.0.0.1:57580 (error 108)
Metrics: {"server-negotiator.queue_time_us":51}
W20260806 18:34:35.500339 20232 init.cc:300] Could not find kerberos principal in credential cache '/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786041246094603-20232-0/krb5kdc/krb5cc' of type FILE
Aug 06 18:34:35 dist-test-slave-1xrh krb5kdc[26016](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041275, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 18:34:35.501255 20232 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Aug 06 18:34:35 dist-test-slave-1xrh krb5kdc[26016](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041275, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 18:34:35.502079 20232 init.cc:295] Successfully reacquired a new kerberos TGT
Aug 06 18:34:35 dist-test-slave-1xrh krb5kdc[26016](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041275, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.19.194.1@KRBTEST.COM
I20260806 18:34:35.510233 26536 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:34:35.502845 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57592 (local address 127.19.194.1:35847)
0806 18:34:35.502962 (+ 117us) server_negotiation.cc:206] Beginning negotiation
0806 18:34:35.502967 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 18:34:35.503057 (+ 90us) server_negotiation.cc:407] Connection header received
0806 18:34:35.503218 (+ 161us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:34:35.503220 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:34:35.503272 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:34:35.503339 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:34:35.503825 (+ 486us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:34:35.504266 (+ 441us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:34:35.505063 (+ 797us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:34:35.505313 (+ 250us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:34:35.505395 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:34:35.507397 (+ 2002us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:34:35.507400 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:34:35.507401 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 18:34:35.507421 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:34:35.508733 (+ 1312us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:34:35.509147 (+ 414us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:34:35.509150 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:34:35.509152 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:34:35.509229 (+ 77us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:34:35.509501 (+ 272us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:34:35.509504 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:34:35.509505 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:34:35.509657 (+ 152us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:34:35.509782 (+ 125us) server_negotiation.cc:1035] Waiting for connection context
0806 18:34:35.510008 (+ 226us) server_negotiation.cc:299] Negotiation successful
0806 18:34:35.510103 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260806 18:34:35.511160 20232 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJW69tn/build/debug/bin/kudu with pid 26125
I20260806 18:34:35.519467 20232 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJW69tn/build/debug/bin/kudu with pid 26263
I20260806 18:34:35.526535 20232 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJW69tn/build/debug/bin/kudu with pid 26398
I20260806 18:34:35.532902 20232 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJW69tn/build/debug/bin/kudu with pid 26029
2026-08-06T18:34:35Z chronyd exiting
[ OK ] SecurityITest.TestCorruptKerberosCC (2172 ms)
[ RUN ] SecurityITest.TestNonDefaultPrincipalMultipleMaster
Loading random data
Initializing database '/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786041246094603-20232-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 18:34:35 dist-test-slave-1xrh krb5kdc[26549](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 18:34:35 dist-test-slave-1xrh krb5kdc[26549](info): set up 2 sockets
Aug 06 18:34:35 dist-test-slave-1xrh krb5kdc[26549](info): commencing operation
krb5kdc: starting...
W20260806 18:34:36.589907 20232 mini_kdc.cc:121] Time spent starting KDC: real 1.034s user 0.005s sys 0.001s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786041246094603-20232-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786041246094603-20232-0/krb5kdc/test-user.keytab.
Aug 06 18:34:36 dist-test-slave-1xrh krb5kdc[26549](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041276, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T18:34:36Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T18:34:36Z Disabled control of system clock
WARNING: no policy specified for oryx/127.19.194.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.19.194.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.19.194.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786041246094603-20232-0/krb5kdc/oryx_127.19.194.62.keytab.
Entry for principal oryx/127.19.194.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786041246094603-20232-0/krb5kdc/oryx_127.19.194.62.keytab.
I20260806 18:34:36.709887 20232 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJW69tn/build/debug/bin/kudu
/tmp/dist-test-taskJW69tn/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786041246094603-20232-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786041246094603-20232-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786041246094603-20232-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786041246094603-20232-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.19.194.62:46737
--webserver_interface=127.19.194.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786041246094603-20232-0/krb5kdc/oryx_127.19.194.62.keytab
--principal=oryx/127.19.194.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.19.194.20:44897
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.19.194.62:46737
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786041246094603-20232-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786041246094603-20232-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786041246094603-20232-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 18:34:36.817472 26563 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 18:34:36.817744 26563 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 18:34:36.817812 26563 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 18:34:36.821693 26563 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 18:34:36.821771 26563 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 18:34:36.821810 26563 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 18:34:36.821864 26563 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 18:34:36.821910 26563 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 18:34:36.826859 26563 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.194.20:44897
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786041246094603-20232-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786041246094603-20232-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.19.194.62:46737
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786041246094603-20232-0/krb5kdc/oryx_127.19.194.62.keytab
--principal=oryx/127.19.194.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.19.194.62:46737
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786041246094603-20232-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.19.194.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26563
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786041246094603-20232-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 17:59:13 UTC on ccycloud.quasar-wvsnzx.root.comops.site
build id 4571028
I20260806 18:34:36.827844 26563 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 18:34:36.828804 26563 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 18:34:36.836830 26568 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 18:34:36.836887 26563 server_base.cc:1034] running on GCE node
W20260806 18:34:36.836813 26569 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:34:36.837077 26571 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 18:34:36.837580 26563 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 18:34:36.838603 26563 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 18:34:36.839823 26563 hybrid_clock.cc:648] HybridClock initialized: now 1786041276839802 us; error 30 us; skew 500 ppm
Aug 06 18:34:36 dist-test-slave-1xrh krb5kdc[26549](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041276, etypes {rep=17 tkt=17 ses=17}, oryx/127.19.194.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 18:34:36.842597 26563 init.cc:377] Logged in from keytab as oryx/127.19.194.62@KRBTEST.COM (short username oryx)
I20260806 18:34:36.843492 26563 webserver.cc:460] Webserver started at http://127.19.194.62:33861/ using document root <none> and password file <none>
I20260806 18:34:36.844039 26563 fs_manager.cc:359] Metadata directory not provided
I20260806 18:34:36.844131 26563 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 18:34:36.844362 26563 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 18:34:36.846346 26563 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786041246094603-20232-0/minicluster-data/master-0/data/instance:
uuid: "a475b841d65e4ef59cb6d53d3e454091"
format_stamp: "Formatted at 2026-08-06 18:34:36 on dist-test-slave-1xrh"
server_key: "bff76b1707d2dbd38dfe893e4003ef71"
server_key_iv: "d92063c2cbe5b1367c2566b8705b7ac9"
server_key_version: "encryptionkey@0"
I20260806 18:34:36.846916 26563 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786041246094603-20232-0/minicluster-data/master-0/wal/instance:
uuid: "a475b841d65e4ef59cb6d53d3e454091"
format_stamp: "Formatted at 2026-08-06 18:34:36 on dist-test-slave-1xrh"
server_key: "bff76b1707d2dbd38dfe893e4003ef71"
server_key_iv: "d92063c2cbe5b1367c2566b8705b7ac9"
server_key_version: "encryptionkey@0"
I20260806 18:34:36.850435 26563 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.002s sys 0.000s
I20260806 18:34:36.852766 26578 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 18:34:36.853879 26563 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260806 18:34:36.854023 26563 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786041246094603-20232-0/minicluster-data/master-0/data,/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786041246094603-20232-0/minicluster-data/master-0/wal
uuid: "a475b841d65e4ef59cb6d53d3e454091"
format_stamp: "Formatted at 2026-08-06 18:34:36 on dist-test-slave-1xrh"
server_key: "bff76b1707d2dbd38dfe893e4003ef71"
server_key_iv: "d92063c2cbe5b1367c2566b8705b7ac9"
server_key_version: "encryptionkey@0"
I20260806 18:34:36.854133 26563 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786041246094603-20232-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786041246094603-20232-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786041246094603-20232-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 18:34:36.876698 26563 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 18:34:36.879976 26563 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 18:34:36.880182 26563 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 18:34:36.887269 26563 rpc_server.cc:307] RPC server started. Bound to: 127.19.194.62:46737
I20260806 18:34:36.887548 26632 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.194.62:46737 every 8 connection(s)
I20260806 18:34:36.888217 26563 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786041246094603-20232-0/minicluster-data/master-0/data/info.pb
I20260806 18:34:36.890832 26633 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 18:34:36.895843 20232 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJW69tn/build/debug/bin/kudu as pid 26563
I20260806 18:34:36.895972 20232 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786041246094603-20232-0/minicluster-data/master-0/wal/instance
I20260806 18:34:36.896310 20232 external_mini_cluster.cc:1428] Setting key 95dd413d2df8f1f9a7d4a3146a29c55b
I20260806 18:34:36.897081 26633 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a475b841d65e4ef59cb6d53d3e454091: Bootstrap starting.
I20260806 18:34:36.899489 26633 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P a475b841d65e4ef59cb6d53d3e454091: Neither blocks nor log segments found. Creating new log.
I20260806 18:34:36.900434 26633 log.cc:824] T 00000000000000000000000000000000 P a475b841d65e4ef59cb6d53d3e454091: Log is configured to *not* fsync() on all Append() calls
I20260806 18:34:36.902544 26633 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a475b841d65e4ef59cb6d53d3e454091: No bootstrap required, opened a new log
Aug 06 18:34:36 dist-test-slave-1xrh krb5kdc[26549](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041276, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.19.194.62@KRBTEST.COM
I20260806 18:34:36.906971 26633 raft_consensus.cc:348] T 00000000000000000000000000000000 P a475b841d65e4ef59cb6d53d3e454091 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a475b841d65e4ef59cb6d53d3e454091" member_type: VOTER last_known_addr { host: "127.19.194.62" port: 46737 } }
I20260806 18:34:36.907240 26633 raft_consensus.cc:374] T 00000000000000000000000000000000 P a475b841d65e4ef59cb6d53d3e454091 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 18:34:36.907315 26633 raft_consensus.cc:729] T 00000000000000000000000000000000 P a475b841d65e4ef59cb6d53d3e454091 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a475b841d65e4ef59cb6d53d3e454091, State: Initialized, Role: FOLLOWER
I20260806 18:34:36.907718 26633 consensus_queue.cc:260] T 00000000000000000000000000000000 P a475b841d65e4ef59cb6d53d3e454091 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a475b841d65e4ef59cb6d53d3e454091" member_type: VOTER last_known_addr { host: "127.19.194.62" port: 46737 } }
I20260806 18:34:36.907853 26633 raft_consensus.cc:388] T 00000000000000000000000000000000 P a475b841d65e4ef59cb6d53d3e454091 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 18:34:36.907941 26633 raft_consensus.cc:482] T 00000000000000000000000000000000 P a475b841d65e4ef59cb6d53d3e454091 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 18:34:36.908042 26633 raft_consensus.cc:3037] T 00000000000000000000000000000000 P a475b841d65e4ef59cb6d53d3e454091 [term 0 FOLLOWER]: Advancing to term 1
I20260806 18:34:36.909205 26633 raft_consensus.cc:504] T 00000000000000000000000000000000 P a475b841d65e4ef59cb6d53d3e454091 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a475b841d65e4ef59cb6d53d3e454091" member_type: VOTER last_known_addr { host: "127.19.194.62" port: 46737 } }
I20260806 18:34:36.909560 26633 leader_election.cc:302] T 00000000000000000000000000000000 P a475b841d65e4ef59cb6d53d3e454091 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: a475b841d65e4ef59cb6d53d3e454091; no voters:
I20260806 18:34:36.909920 26633 leader_election.cc:288] T 00000000000000000000000000000000 P a475b841d65e4ef59cb6d53d3e454091 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 18:34:36.910101 26638 raft_consensus.cc:2781] T 00000000000000000000000000000000 P a475b841d65e4ef59cb6d53d3e454091 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 18:34:36.910356 26638 raft_consensus.cc:686] T 00000000000000000000000000000000 P a475b841d65e4ef59cb6d53d3e454091 [term 1 LEADER]: Becoming Leader. State: Replica: a475b841d65e4ef59cb6d53d3e454091, State: Running, Role: LEADER
I20260806 18:34:36.910707 26638 consensus_queue.cc:237] T 00000000000000000000000000000000 P a475b841d65e4ef59cb6d53d3e454091 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a475b841d65e4ef59cb6d53d3e454091" member_type: VOTER last_known_addr { host: "127.19.194.62" port: 46737 } }
I20260806 18:34:36.911271 26633 sys_catalog.cc:564] T 00000000000000000000000000000000 P a475b841d65e4ef59cb6d53d3e454091 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 18:34:36.911378 26636 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:34:36.897689 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44400 (local address 127.19.194.62:46737)
0806 18:34:36.898086 (+ 397us) server_negotiation.cc:206] Beginning negotiation
0806 18:34:36.898105 (+ 19us) server_negotiation.cc:399] Waiting for connection header
0806 18:34:36.898148 (+ 43us) server_negotiation.cc:407] Connection header received
0806 18:34:36.898778 (+ 630us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:34:36.898793 (+ 15us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:34:36.899107 (+ 314us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:34:36.899407 (+ 300us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:34:36.900330 (+ 923us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:34:36.901338 (+ 1008us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:34:36.902490 (+ 1152us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:34:36.903000 (+ 510us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:34:36.903115 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:34:36.906098 (+ 2983us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:34:36.906103 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:34:36.906116 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 18:34:36.906141 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:34:36.908204 (+ 2063us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:34:36.908682 (+ 478us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:34:36.908686 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:34:36.908690 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:34:36.908794 (+ 104us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:34:36.909080 (+ 286us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:34:36.909082 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:34:36.909083 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:34:36.909533 (+ 450us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:34:36.909709 (+ 176us) server_negotiation.cc:1035] Waiting for connection context
0806 18:34:36.910041 (+ 332us) server_negotiation.cc:299] Negotiation successful
0806 18:34:36.910353 (+ 312us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":262,"thread_start_us":137,"threads_started":1}
I20260806 18:34:36.911926 26638 sys_catalog.cc:455] T 00000000000000000000000000000000 P a475b841d65e4ef59cb6d53d3e454091 [sys.catalog]: SysCatalogTable state changed. Reason: New leader a475b841d65e4ef59cb6d53d3e454091. Latest consensus state: current_term: 1 leader_uuid: "a475b841d65e4ef59cb6d53d3e454091" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a475b841d65e4ef59cb6d53d3e454091" member_type: VOTER last_known_addr { host: "127.19.194.62" port: 46737 } } }
I20260806 18:34:36.912055 26638 sys_catalog.cc:458] T 00000000000000000000000000000000 P a475b841d65e4ef59cb6d53d3e454091 [sys.catalog]: This master's current role is: LEADER
I20260806 18:34:36.914166 26639 sys_catalog.cc:455] T 00000000000000000000000000000000 P a475b841d65e4ef59cb6d53d3e454091 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "a475b841d65e4ef59cb6d53d3e454091" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a475b841d65e4ef59cb6d53d3e454091" member_type: VOTER last_known_addr { host: "127.19.194.62" port: 46737 } } }
I20260806 18:34:36.914275 26639 sys_catalog.cc:458] T 00000000000000000000000000000000 P a475b841d65e4ef59cb6d53d3e454091 [sys.catalog]: This master's current role is: LEADER
W20260806 18:34:36.916079 26653 catalog_manager.cc:1558] T 00000000000000000000000000000000 P a475b841d65e4ef59cb6d53d3e454091: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260806 18:34:36.916146 26653 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260806 18:34:36.916226 26644 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 18:34:36.916931 26644 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 18:34:36.922050 26644 catalog_manager.cc:1347] Generated new cluster ID: 3fbbab1f31df424cb2df0f828bded742
I20260806 18:34:36.922114 26644 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 18:34:36.928314 26644 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 18:34:36.929147 26644 catalog_manager.cc:1504] Loading token signing keys...
I20260806 18:34:36.938984 26644 catalog_manager.cc:5868] T 00000000000000000000000000000000 P a475b841d65e4ef59cb6d53d3e454091: Generated new TSK 0
I20260806 18:34:36.939635 26644 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for oryx/127.19.194.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.19.194.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.19.194.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786041246094603-20232-0/krb5kdc/oryx_127.19.194.1.keytab.
Entry for principal oryx/127.19.194.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786041246094603-20232-0/krb5kdc/oryx_127.19.194.1.keytab.
I20260806 18:34:36.982285 20232 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJW69tn/build/debug/bin/kudu
/tmp/dist-test-taskJW69tn/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786041246094603-20232-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786041246094603-20232-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786041246094603-20232-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786041246094603-20232-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.19.194.1:0
--local_ip_for_outbound_sockets=127.19.194.1
--webserver_interface=127.19.194.1
--webserver_port=0
--tserver_master_addrs=127.19.194.62:46737
--keytab_file=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786041246094603-20232-0/krb5kdc/oryx_127.19.194.1.keytab
--principal=oryx/127.19.194.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.19.194.20:44897
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786041246094603-20232-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786041246094603-20232-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786041246094603-20232-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 18:34:37.096077 26659 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 18:34:37.096493 26659 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 18:34:37.096571 26659 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 18:34:37.100677 26659 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 18:34:37.100752 26659 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 18:34:37.100848 26659 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.19.194.1
I20260806 18:34:37.106014 26659 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.194.20:44897
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786041246094603-20232-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786041246094603-20232-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786041246094603-20232-0/krb5kdc/oryx_127.19.194.1.keytab
--principal=oryx/127.19.194.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.19.194.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786041246094603-20232-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.19.194.1
--webserver_port=0
--tserver_master_addrs=127.19.194.62:46737
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26659
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.19.194.1
--log_dir=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786041246094603-20232-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 17:59:13 UTC on ccycloud.quasar-wvsnzx.root.comops.site
build id 4571028
I20260806 18:34:37.106981 26659 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 18:34:37.108057 26659 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 18:34:37.117251 26667 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:34:37.117317 26664 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 18:34:37.117453 26659 server_base.cc:1034] running on GCE node
W20260806 18:34:37.117589 26665 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 18:34:37.118007 26659 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 18:34:37.118741 26659 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 18:34:37.119916 26659 hybrid_clock.cc:648] HybridClock initialized: now 1786041277119887 us; error 38 us; skew 500 ppm
Aug 06 18:34:37 dist-test-slave-1xrh krb5kdc[26549](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041277, etypes {rep=17 tkt=17 ses=17}, oryx/127.19.194.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 18:34:37.122771 26659 init.cc:377] Logged in from keytab as oryx/127.19.194.1@KRBTEST.COM (short username oryx)
I20260806 18:34:37.123719 26659 webserver.cc:460] Webserver started at http://127.19.194.1:33093/ using document root <none> and password file <none>
I20260806 18:34:37.124336 26659 fs_manager.cc:359] Metadata directory not provided
I20260806 18:34:37.124409 26659 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 18:34:37.124662 26659 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 18:34:37.126726 26659 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786041246094603-20232-0/minicluster-data/ts-0/data/instance:
uuid: "a7dbdd68b24540cda5ada7e1e0d73ccd"
format_stamp: "Formatted at 2026-08-06 18:34:37 on dist-test-slave-1xrh"
server_key: "d129966921ed58c57baacbf98cc8e63e"
server_key_iv: "62b325fd36ca0563239daa6ce37a24f1"
server_key_version: "encryptionkey@0"
I20260806 18:34:37.127406 26659 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786041246094603-20232-0/minicluster-data/ts-0/wal/instance:
uuid: "a7dbdd68b24540cda5ada7e1e0d73ccd"
format_stamp: "Formatted at 2026-08-06 18:34:37 on dist-test-slave-1xrh"
server_key: "d129966921ed58c57baacbf98cc8e63e"
server_key_iv: "62b325fd36ca0563239daa6ce37a24f1"
server_key_version: "encryptionkey@0"
I20260806 18:34:37.131261 26659 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.000s
I20260806 18:34:37.133649 26674 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 18:34:37.134888 26659 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260806 18:34:37.134986 26659 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786041246094603-20232-0/minicluster-data/ts-0/data,/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786041246094603-20232-0/minicluster-data/ts-0/wal
uuid: "a7dbdd68b24540cda5ada7e1e0d73ccd"
format_stamp: "Formatted at 2026-08-06 18:34:37 on dist-test-slave-1xrh"
server_key: "d129966921ed58c57baacbf98cc8e63e"
server_key_iv: "62b325fd36ca0563239daa6ce37a24f1"
server_key_version: "encryptionkey@0"
I20260806 18:34:37.135219 26659 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786041246094603-20232-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786041246094603-20232-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786041246094603-20232-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 18:34:37.158288 26659 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 18:34:37.161196 26659 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 18:34:37.161425 26659 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 18:34:37.162011 26659 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 18:34:37.162824 26659 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 18:34:37.162871 26659 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 18:34:37.162942 26659 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 18:34:37.162974 26659 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 18:34:37.172519 26659 rpc_server.cc:307] RPC server started. Bound to: 127.19.194.1:33871
I20260806 18:34:37.172545 26789 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.194.1:33871 every 8 connection(s)
I20260806 18:34:37.173418 26659 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786041246094603-20232-0/minicluster-data/ts-0/data/info.pb
I20260806 18:34:37.179397 20232 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJW69tn/build/debug/bin/kudu as pid 26659
I20260806 18:34:37.179497 20232 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786041246094603-20232-0/minicluster-data/ts-0/wal/instance
I20260806 18:34:37.179797 20232 external_mini_cluster.cc:1428] Setting key fb03bc430bc772ef5180e1d3a6e2cc14
Aug 06 18:34:37 dist-test-slave-1xrh krb5kdc[26549](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041277, etypes {rep=17 tkt=17 ses=17}, oryx/127.19.194.1@KRBTEST.COM for oryx/127.19.194.62@KRBTEST.COM
I20260806 18:34:37.189105 26636 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:34:37.175227 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.194.1:33597 (local address 127.19.194.62:46737)
0806 18:34:37.175368 (+ 141us) server_negotiation.cc:206] Beginning negotiation
0806 18:34:37.175372 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 18:34:37.176306 (+ 934us) server_negotiation.cc:407] Connection header received
0806 18:34:37.177827 (+ 1521us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:34:37.177830 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:34:37.177905 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:34:37.178022 (+ 117us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:34:37.179264 (+ 1242us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:34:37.179934 (+ 670us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:34:37.181455 (+ 1521us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:34:37.181699 (+ 244us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:34:37.181781 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:34:37.184942 (+ 3161us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:34:37.184949 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:34:37.184955 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 18:34:37.184989 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:34:37.186790 (+ 1801us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:34:37.187376 (+ 586us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:34:37.187382 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:34:37.187386 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:34:37.187501 (+ 115us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:34:37.187858 (+ 357us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:34:37.187864 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:34:37.187867 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:34:37.188134 (+ 267us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:34:37.188265 (+ 131us) server_negotiation.cc:1035] Waiting for connection context
0806 18:34:37.188727 (+ 462us) server_negotiation.cc:299] Negotiation successful
0806 18:34:37.188885 (+ 158us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260806 18:34:37.189698 26792 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:34:37.175590 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.194.62:46737 (local address 127.19.194.1:33597)
0806 18:34:37.176058 (+ 468us) negotiation.cc:107] Waiting for socket to connect
0806 18:34:37.176101 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0806 18:34:37.177599 (+ 1498us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:34:37.178203 (+ 604us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:34:37.178210 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:34:37.178572 (+ 362us) client_negotiation.cc:189] Negotiated authn=SASL
0806 18:34:37.179043 (+ 471us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:34:37.179057 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:34:37.180069 (+ 1012us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:34:37.180074 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:34:37.181318 (+ 1244us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:34:37.181327 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:34:37.181872 (+ 545us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:34:37.181879 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:34:37.182130 (+ 251us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:34:37.182706 (+ 576us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 18:34:37.182733 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0806 18:34:37.184770 (+ 2037us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 18:34:37.186919 (+ 2149us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:34:37.186924 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:34:37.186934 (+ 10us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:34:37.187256 (+ 322us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:34:37.187603 (+ 347us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:34:37.187605 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:34:37.187606 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:34:37.187737 (+ 131us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:34:37.188259 (+ 522us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 18:34:37.188265 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 18:34:37.188475 (+ 210us) client_negotiation.cc:769] Sending connection context
0806 18:34:37.188621 (+ 146us) client_negotiation.cc:240] Negotiation successful
0806 18:34:37.188855 (+ 234us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":284,"thread_start_us":111,"threads_started":1}
I20260806 18:34:37.190979 26790 heartbeater.cc:347] Connected to a master server at 127.19.194.62:46737
WARNING: no policy specified for oryx/127.19.194.2@KRBTEST.COM; defaulting to no policy
I20260806 18:34:37.191272 26790 heartbeater.cc:464] Registering TS with master...
I20260806 18:34:37.191804 26790 heartbeater.cc:511] Master 127.19.194.62:46737 requested a full tablet report, sending...
I20260806 18:34:37.193351 26597 ts_manager.cc:194] Registered new tserver with Master: a7dbdd68b24540cda5ada7e1e0d73ccd (127.19.194.1:33871)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.19.194.2@KRBTEST.COM" created.
I20260806 18:34:37.194610 26597 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.19.194.1@KRBTEST.COM'} at 127.19.194.1:33597
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.19.194.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786041246094603-20232-0/krb5kdc/oryx_127.19.194.2.keytab.
Entry for principal oryx/127.19.194.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786041246094603-20232-0/krb5kdc/oryx_127.19.194.2.keytab.
I20260806 18:34:37.209676 20232 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJW69tn/build/debug/bin/kudu
/tmp/dist-test-taskJW69tn/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786041246094603-20232-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786041246094603-20232-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786041246094603-20232-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786041246094603-20232-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.19.194.2:0
--local_ip_for_outbound_sockets=127.19.194.2
--webserver_interface=127.19.194.2
--webserver_port=0
--tserver_master_addrs=127.19.194.62:46737
--keytab_file=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786041246094603-20232-0/krb5kdc/oryx_127.19.194.2.keytab
--principal=oryx/127.19.194.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.19.194.20:44897
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786041246094603-20232-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786041246094603-20232-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786041246094603-20232-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 18:34:37.321769 26795 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 18:34:37.322016 26795 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 18:34:37.322057 26795 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 18:34:37.326195 26795 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 18:34:37.326246 26795 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 18:34:37.326300 26795 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.19.194.2
I20260806 18:34:37.331837 26795 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.194.20:44897
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786041246094603-20232-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786041246094603-20232-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786041246094603-20232-0/krb5kdc/oryx_127.19.194.2.keytab
--principal=oryx/127.19.194.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.19.194.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786041246094603-20232-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.19.194.2
--webserver_port=0
--tserver_master_addrs=127.19.194.62:46737
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26795
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.19.194.2
--log_dir=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786041246094603-20232-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 17:59:13 UTC on ccycloud.quasar-wvsnzx.root.comops.site
build id 4571028
I20260806 18:34:37.332818 26795 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 18:34:37.333796 26795 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 18:34:37.343012 26800 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:34:37.343043 26803 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 18:34:37.343163 26795 server_base.cc:1034] running on GCE node
W20260806 18:34:37.343070 26801 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 18:34:37.343623 26795 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 18:34:37.344269 26795 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 18:34:37.345454 26795 hybrid_clock.cc:648] HybridClock initialized: now 1786041277345445 us; error 29 us; skew 500 ppm
Aug 06 18:34:37 dist-test-slave-1xrh krb5kdc[26549](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041277, etypes {rep=17 tkt=17 ses=17}, oryx/127.19.194.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 18:34:37.348228 26795 init.cc:377] Logged in from keytab as oryx/127.19.194.2@KRBTEST.COM (short username oryx)
I20260806 18:34:37.349063 26795 webserver.cc:460] Webserver started at http://127.19.194.2:35523/ using document root <none> and password file <none>
I20260806 18:34:37.349598 26795 fs_manager.cc:359] Metadata directory not provided
I20260806 18:34:37.349690 26795 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 18:34:37.349917 26795 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 18:34:37.351882 26795 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786041246094603-20232-0/minicluster-data/ts-1/data/instance:
uuid: "a5630d59c6324f3b87d9d2a1f4024c45"
format_stamp: "Formatted at 2026-08-06 18:34:37 on dist-test-slave-1xrh"
server_key: "2c031e1e1419e1611cd5cef94f10e964"
server_key_iv: "e938fb4774cac2c1c426a6f635e5104b"
server_key_version: "encryptionkey@0"
I20260806 18:34:37.352459 26795 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786041246094603-20232-0/minicluster-data/ts-1/wal/instance:
uuid: "a5630d59c6324f3b87d9d2a1f4024c45"
format_stamp: "Formatted at 2026-08-06 18:34:37 on dist-test-slave-1xrh"
server_key: "2c031e1e1419e1611cd5cef94f10e964"
server_key_iv: "e938fb4774cac2c1c426a6f635e5104b"
server_key_version: "encryptionkey@0"
I20260806 18:34:37.355959 26795 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20260806 18:34:37.358297 26810 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 18:34:37.359537 26795 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260806 18:34:37.359669 26795 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786041246094603-20232-0/minicluster-data/ts-1/data,/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786041246094603-20232-0/minicluster-data/ts-1/wal
uuid: "a5630d59c6324f3b87d9d2a1f4024c45"
format_stamp: "Formatted at 2026-08-06 18:34:37 on dist-test-slave-1xrh"
server_key: "2c031e1e1419e1611cd5cef94f10e964"
server_key_iv: "e938fb4774cac2c1c426a6f635e5104b"
server_key_version: "encryptionkey@0"
I20260806 18:34:37.359782 26795 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786041246094603-20232-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786041246094603-20232-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786041246094603-20232-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 18:34:37.385097 26795 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 18:34:37.387864 26795 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 18:34:37.388068 26795 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 18:34:37.388700 26795 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 18:34:37.389525 26795 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 18:34:37.389599 26795 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 18:34:37.389665 26795 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 18:34:37.389711 26795 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 18:34:37.398542 26795 rpc_server.cc:307] RPC server started. Bound to: 127.19.194.2:44715
I20260806 18:34:37.398583 26927 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.194.2:44715 every 8 connection(s)
I20260806 18:34:37.399447 26795 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786041246094603-20232-0/minicluster-data/ts-1/data/info.pb
I20260806 18:34:37.405849 20232 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJW69tn/build/debug/bin/kudu as pid 26795
I20260806 18:34:37.405936 20232 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786041246094603-20232-0/minicluster-data/ts-1/wal/instance
I20260806 18:34:37.406219 20232 external_mini_cluster.cc:1428] Setting key 062934343e33cb4b36ffe4d3653ac34e
Aug 06 18:34:37 dist-test-slave-1xrh krb5kdc[26549](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041277, etypes {rep=17 tkt=17 ses=17}, oryx/127.19.194.2@KRBTEST.COM for oryx/127.19.194.62@KRBTEST.COM
I20260806 18:34:37.413475 26636 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:34:37.401130 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.194.2:36165 (local address 127.19.194.62:46737)
0806 18:34:37.401263 (+ 133us) server_negotiation.cc:206] Beginning negotiation
0806 18:34:37.401267 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 18:34:37.402130 (+ 863us) server_negotiation.cc:407] Connection header received
0806 18:34:37.403380 (+ 1250us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:34:37.403383 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:34:37.403431 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:34:37.403504 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:34:37.404602 (+ 1098us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:34:37.405109 (+ 507us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:34:37.406277 (+ 1168us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:34:37.406556 (+ 279us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:34:37.406687 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:34:37.409403 (+ 2716us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:34:37.409410 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:34:37.409416 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 18:34:37.409452 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:34:37.410940 (+ 1488us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:34:37.411627 (+ 687us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:34:37.411632 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:34:37.411634 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:34:37.411747 (+ 113us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:34:37.412189 (+ 442us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:34:37.412192 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:34:37.412194 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:34:37.412507 (+ 313us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:34:37.412614 (+ 107us) server_negotiation.cc:1035] Waiting for connection context
0806 18:34:37.413169 (+ 555us) server_negotiation.cc:299] Negotiation successful
0806 18:34:37.413302 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260806 18:34:37.414602 26930 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:34:37.401508 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.194.62:46737 (local address 127.19.194.2:36165)
0806 18:34:37.401946 (+ 438us) negotiation.cc:107] Waiting for socket to connect
0806 18:34:37.401989 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0806 18:34:37.403199 (+ 1210us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:34:37.403661 (+ 462us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:34:37.403667 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:34:37.404030 (+ 363us) client_negotiation.cc:189] Negotiated authn=SASL
0806 18:34:37.404479 (+ 449us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:34:37.404492 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:34:37.405220 (+ 728us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:34:37.405223 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:34:37.406139 (+ 916us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:34:37.406149 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:34:37.406739 (+ 590us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:34:37.406743 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:34:37.406936 (+ 193us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:34:37.407577 (+ 641us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 18:34:37.407601 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0806 18:34:37.409228 (+ 1627us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 18:34:37.411102 (+ 1874us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:34:37.411108 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:34:37.411145 (+ 37us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:34:37.411511 (+ 366us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:34:37.411849 (+ 338us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:34:37.411852 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:34:37.411854 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:34:37.412075 (+ 221us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:34:37.412629 (+ 554us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 18:34:37.412637 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 18:34:37.412890 (+ 253us) client_negotiation.cc:769] Sending connection context
0806 18:34:37.413080 (+ 190us) client_negotiation.cc:240] Negotiation successful
0806 18:34:37.413427 (+ 347us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":257,"thread_start_us":113,"threads_started":1}
I20260806 18:34:37.415859 26928 heartbeater.cc:347] Connected to a master server at 127.19.194.62:46737
I20260806 18:34:37.416114 26928 heartbeater.cc:464] Registering TS with master...
I20260806 18:34:37.416713 26928 heartbeater.cc:511] Master 127.19.194.62:46737 requested a full tablet report, sending...
WARNING: no policy specified for oryx/127.19.194.3@KRBTEST.COM; defaulting to no policy
I20260806 18:34:37.417819 26597 ts_manager.cc:194] Registered new tserver with Master: a5630d59c6324f3b87d9d2a1f4024c45 (127.19.194.2:44715)
I20260806 18:34:37.418712 26597 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.19.194.2@KRBTEST.COM'} at 127.19.194.2:36165
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.19.194.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.19.194.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786041246094603-20232-0/krb5kdc/oryx_127.19.194.3.keytab.
Entry for principal oryx/127.19.194.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786041246094603-20232-0/krb5kdc/oryx_127.19.194.3.keytab.
I20260806 18:34:37.434721 20232 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJW69tn/build/debug/bin/kudu
/tmp/dist-test-taskJW69tn/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786041246094603-20232-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786041246094603-20232-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786041246094603-20232-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786041246094603-20232-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.19.194.3:0
--local_ip_for_outbound_sockets=127.19.194.3
--webserver_interface=127.19.194.3
--webserver_port=0
--tserver_master_addrs=127.19.194.62:46737
--keytab_file=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786041246094603-20232-0/krb5kdc/oryx_127.19.194.3.keytab
--principal=oryx/127.19.194.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.19.194.20:44897
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786041246094603-20232-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786041246094603-20232-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786041246094603-20232-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 18:34:37.542295 26933 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 18:34:37.542517 26933 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 18:34:37.542586 26933 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 18:34:37.546617 26933 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 18:34:37.546671 26933 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 18:34:37.546783 26933 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.19.194.3
I20260806 18:34:37.551916 26933 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.194.20:44897
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786041246094603-20232-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786041246094603-20232-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786041246094603-20232-0/krb5kdc/oryx_127.19.194.3.keytab
--principal=oryx/127.19.194.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.19.194.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786041246094603-20232-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.19.194.3
--webserver_port=0
--tserver_master_addrs=127.19.194.62:46737
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26933
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.19.194.3
--log_dir=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786041246094603-20232-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 17:59:13 UTC on ccycloud.quasar-wvsnzx.root.comops.site
build id 4571028
I20260806 18:34:37.552843 26933 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 18:34:37.553779 26933 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 18:34:37.563292 26941 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:34:37.563328 26938 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:34:37.563310 26939 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 18:34:37.563603 26933 server_base.cc:1034] running on GCE node
I20260806 18:34:37.564095 26933 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 18:34:37.564756 26933 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 18:34:37.565944 26933 hybrid_clock.cc:648] HybridClock initialized: now 1786041277565911 us; error 45 us; skew 500 ppm
Aug 06 18:34:37 dist-test-slave-1xrh krb5kdc[26549](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041277, etypes {rep=17 tkt=17 ses=17}, oryx/127.19.194.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 18:34:37.568746 26933 init.cc:377] Logged in from keytab as oryx/127.19.194.3@KRBTEST.COM (short username oryx)
I20260806 18:34:37.569571 26933 webserver.cc:460] Webserver started at http://127.19.194.3:46369/ using document root <none> and password file <none>
I20260806 18:34:37.570179 26933 fs_manager.cc:359] Metadata directory not provided
I20260806 18:34:37.570282 26933 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 18:34:37.570528 26933 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 18:34:37.572609 26933 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786041246094603-20232-0/minicluster-data/ts-2/data/instance:
uuid: "402d0dd8fe1f4b4385b0b94ba213aaa6"
format_stamp: "Formatted at 2026-08-06 18:34:37 on dist-test-slave-1xrh"
server_key: "f3b08bfcc4dc3c1fbe6665eb0d24ecda"
server_key_iv: "deb4eabedf15d2d6658505ee6c07da13"
server_key_version: "encryptionkey@0"
I20260806 18:34:37.573195 26933 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786041246094603-20232-0/minicluster-data/ts-2/wal/instance:
uuid: "402d0dd8fe1f4b4385b0b94ba213aaa6"
format_stamp: "Formatted at 2026-08-06 18:34:37 on dist-test-slave-1xrh"
server_key: "f3b08bfcc4dc3c1fbe6665eb0d24ecda"
server_key_iv: "deb4eabedf15d2d6658505ee6c07da13"
server_key_version: "encryptionkey@0"
I20260806 18:34:37.576939 26933 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.005s sys 0.000s
I20260806 18:34:37.579312 26948 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 18:34:37.580515 26933 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260806 18:34:37.580663 26933 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786041246094603-20232-0/minicluster-data/ts-2/data,/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786041246094603-20232-0/minicluster-data/ts-2/wal
uuid: "402d0dd8fe1f4b4385b0b94ba213aaa6"
format_stamp: "Formatted at 2026-08-06 18:34:37 on dist-test-slave-1xrh"
server_key: "f3b08bfcc4dc3c1fbe6665eb0d24ecda"
server_key_iv: "deb4eabedf15d2d6658505ee6c07da13"
server_key_version: "encryptionkey@0"
I20260806 18:34:37.580773 26933 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786041246094603-20232-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786041246094603-20232-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786041246094603-20232-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 18:34:37.596481 26933 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 18:34:37.599988 26933 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 18:34:37.600200 26933 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 18:34:37.600814 26933 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 18:34:37.601667 26933 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 18:34:37.601739 26933 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 18:34:37.601814 26933 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 18:34:37.601868 26933 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 18:34:37.610703 26933 rpc_server.cc:307] RPC server started. Bound to: 127.19.194.3:42189
I20260806 18:34:37.610759 27064 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.194.3:42189 every 8 connection(s)
I20260806 18:34:37.611737 26933 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786041246094603-20232-0/minicluster-data/ts-2/data/info.pb
I20260806 18:34:37.620694 20232 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJW69tn/build/debug/bin/kudu as pid 26933
I20260806 18:34:37.620855 20232 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786041246094603-20232-0/minicluster-data/ts-2/wal/instance
I20260806 18:34:37.621202 20232 external_mini_cluster.cc:1428] Setting key d99aa1d6eef61635944c4fc1270ec6f0
Aug 06 18:34:37 dist-test-slave-1xrh krb5kdc[26549](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041277, etypes {rep=17 tkt=17 ses=17}, oryx/127.19.194.3@KRBTEST.COM for oryx/127.19.194.62@KRBTEST.COM
I20260806 18:34:37.626077 26636 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:34:37.613711 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.194.3:50915 (local address 127.19.194.62:46737)
0806 18:34:37.613855 (+ 144us) server_negotiation.cc:206] Beginning negotiation
0806 18:34:37.613860 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 18:34:37.614703 (+ 843us) server_negotiation.cc:407] Connection header received
0806 18:34:37.616175 (+ 1472us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:34:37.616178 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:34:37.616241 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:34:37.616348 (+ 107us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:34:37.617504 (+ 1156us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:34:37.618072 (+ 568us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:34:37.619279 (+ 1207us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:34:37.619503 (+ 224us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:34:37.619631 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:34:37.622452 (+ 2821us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:34:37.622456 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:34:37.622459 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 18:34:37.622486 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:34:37.624055 (+ 1569us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:34:37.624561 (+ 506us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:34:37.624564 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:34:37.624566 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:34:37.624650 (+ 84us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:34:37.624993 (+ 343us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:34:37.624998 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:34:37.625000 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:34:37.625187 (+ 187us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:34:37.625297 (+ 110us) server_negotiation.cc:1035] Waiting for connection context
0806 18:34:37.625783 (+ 486us) server_negotiation.cc:299] Negotiation successful
0806 18:34:37.625909 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260806 18:34:37.626849 27067 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:34:37.614096 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.194.62:46737 (local address 127.19.194.3:50915)
0806 18:34:37.614553 (+ 457us) negotiation.cc:107] Waiting for socket to connect
0806 18:34:37.614596 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0806 18:34:37.615949 (+ 1353us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:34:37.616519 (+ 570us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:34:37.616527 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:34:37.616886 (+ 359us) client_negotiation.cc:189] Negotiated authn=SASL
0806 18:34:37.617355 (+ 469us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:34:37.617369 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:34:37.618214 (+ 845us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:34:37.618218 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:34:37.619071 (+ 853us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:34:37.619080 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:34:37.619688 (+ 608us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:34:37.619692 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:34:37.619861 (+ 169us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:34:37.620470 (+ 609us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 18:34:37.620487 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0806 18:34:37.622288 (+ 1801us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 18:34:37.624176 (+ 1888us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:34:37.624182 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:34:37.624197 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:34:37.624451 (+ 254us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:34:37.624746 (+ 295us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:34:37.624750 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:34:37.624752 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:34:37.624895 (+ 143us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:34:37.625297 (+ 402us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 18:34:37.625305 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 18:34:37.625550 (+ 245us) client_negotiation.cc:769] Sending connection context
0806 18:34:37.625741 (+ 191us) client_negotiation.cc:240] Negotiation successful
0806 18:34:37.626065 (+ 324us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":270,"thread_start_us":115,"threads_started":1}
I20260806 18:34:37.627971 27065 heartbeater.cc:347] Connected to a master server at 127.19.194.62:46737
I20260806 18:34:37.628185 27065 heartbeater.cc:464] Registering TS with master...
I20260806 18:34:37.628620 27065 heartbeater.cc:511] Master 127.19.194.62:46737 requested a full tablet report, sending...
I20260806 18:34:37.629541 26597 ts_manager.cc:194] Registered new tserver with Master: 402d0dd8fe1f4b4385b0b94ba213aaa6 (127.19.194.3:42189)
I20260806 18:34:37.630069 26597 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.19.194.3@KRBTEST.COM'} at 127.19.194.3:50915
I20260806 18:34:37.635286 20232 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
WARNING: no policy specified for oryx/127.19.194.61@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.19.194.61@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.19.194.61 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786041246094603-20232-0/krb5kdc/oryx_127.19.194.61.keytab.
Entry for principal oryx/127.19.194.61 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786041246094603-20232-0/krb5kdc/oryx_127.19.194.61.keytab.
I20260806 18:34:37.662081 20232 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJW69tn/build/debug/bin/kudu
/tmp/dist-test-taskJW69tn/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786041246094603-20232-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786041246094603-20232-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786041246094603-20232-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786041246094603-20232-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.19.194.61:39189
--webserver_interface=127.19.194.61
--webserver_port=0
--keytab_file=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786041246094603-20232-0/krb5kdc/oryx_127.19.194.61.keytab
--principal=oryx/127.19.194.61
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.19.194.20:44897
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.19.194.62:46737,127.19.194.61:39189
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786041246094603-20232-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786041246094603-20232-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786041246094603-20232-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 18:34:37.771739 27070 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 18:34:37.771991 27070 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 18:34:37.772070 27070 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 18:34:37.776105 27070 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 18:34:37.776160 27070 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 18:34:37.776178 27070 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 18:34:37.776196 27070 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 18:34:37.776244 27070 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 18:34:37.781481 27070 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.194.20:44897
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786041246094603-20232-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786041246094603-20232-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.19.194.62:46737,127.19.194.61:39189
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786041246094603-20232-0/krb5kdc/oryx_127.19.194.61.keytab
--principal=oryx/127.19.194.61
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.19.194.61:39189
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786041246094603-20232-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.19.194.61
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27070
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786041246094603-20232-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 17:59:13 UTC on ccycloud.quasar-wvsnzx.root.comops.site
build id 4571028
W20260806 18:34:37.781697 27070 master_options.cc:55] Only 2 masters are specified by master_addresses_flag ('127.19.194.62:46737,127.19.194.61:39189'), but minimum of 3 are required to tolerate failures of any one master. It is recommended to use at least 3 masters.
I20260806 18:34:37.782544 27070 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 18:34:37.783612 27070 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 18:34:37.791976 27076 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 18:34:37.792048 27070 server_base.cc:1034] running on GCE node
W20260806 18:34:37.792037 27075 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:34:37.791982 27078 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 18:34:37.792734 27070 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 18:34:37.793787 27070 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 18:34:37.794957 27070 hybrid_clock.cc:648] HybridClock initialized: now 1786041277794928 us; error 43 us; skew 500 ppm
Aug 06 18:34:37 dist-test-slave-1xrh krb5kdc[26549](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041277, etypes {rep=17 tkt=17 ses=17}, oryx/127.19.194.61@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 18:34:37.798170 27070 init.cc:377] Logged in from keytab as oryx/127.19.194.61@KRBTEST.COM (short username oryx)
I20260806 18:34:37.799005 27070 webserver.cc:460] Webserver started at http://127.19.194.61:33173/ using document root <none> and password file <none>
I20260806 18:34:37.799580 27070 fs_manager.cc:359] Metadata directory not provided
I20260806 18:34:37.799644 27070 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 18:34:37.799806 27070 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 18:34:37.801729 27070 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786041246094603-20232-0/minicluster-data/master-1/data/instance:
uuid: "9a38e245bcc34630ba3cc6fb6ca11cb3"
format_stamp: "Formatted at 2026-08-06 18:34:37 on dist-test-slave-1xrh"
server_key: "4c952ddde25c1a62222a1ee034d444b3"
server_key_iv: "b0791910121edd82c0ccc6dc106e0048"
server_key_version: "encryptionkey@0"
I20260806 18:34:37.802291 27070 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786041246094603-20232-0/minicluster-data/master-1/wal/instance:
uuid: "9a38e245bcc34630ba3cc6fb6ca11cb3"
format_stamp: "Formatted at 2026-08-06 18:34:37 on dist-test-slave-1xrh"
server_key: "4c952ddde25c1a62222a1ee034d444b3"
server_key_iv: "b0791910121edd82c0ccc6dc106e0048"
server_key_version: "encryptionkey@0"
I20260806 18:34:37.805905 27070 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.006s sys 0.000s
I20260806 18:34:37.808220 27085 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 18:34:37.809489 27070 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260806 18:34:37.809624 27070 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786041246094603-20232-0/minicluster-data/master-1/data,/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786041246094603-20232-0/minicluster-data/master-1/wal
uuid: "9a38e245bcc34630ba3cc6fb6ca11cb3"
format_stamp: "Formatted at 2026-08-06 18:34:37 on dist-test-slave-1xrh"
server_key: "4c952ddde25c1a62222a1ee034d444b3"
server_key_iv: "b0791910121edd82c0ccc6dc106e0048"
server_key_version: "encryptionkey@0"
I20260806 18:34:37.809731 27070 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786041246094603-20232-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786041246094603-20232-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786041246094603-20232-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 18:34:37.831118 27070 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 18:34:37.833974 27070 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 18:34:37.834316 27070 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 18:34:37.841709 27070 rpc_server.cc:307] RPC server started. Bound to: 127.19.194.61:39189
I20260806 18:34:37.841995 27137 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.194.61:39189 every 8 connection(s)
I20260806 18:34:37.842628 27070 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786041246094603-20232-0/minicluster-data/master-1/data/info.pb
I20260806 18:34:37.845219 27138 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 18:34:37.847782 20232 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJW69tn/build/debug/bin/kudu as pid 27070
I20260806 18:34:37.849056 27138 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.19.194.62" port: 46737 } has no permanent_uuid. Determining permanent_uuid...
Aug 06 18:34:37 dist-test-slave-1xrh krb5kdc[26549](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041277, etypes {rep=17 tkt=17 ses=17}, oryx/127.19.194.61@KRBTEST.COM for oryx/127.19.194.62@KRBTEST.COM
Aug 06 18:34:37 dist-test-slave-1xrh krb5kdc[26549](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041276, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.19.194.61@KRBTEST.COM
I20260806 18:34:37.860790 26636 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:34:37.850862 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44404 (local address 127.19.194.62:46737)
0806 18:34:37.851010 (+ 148us) server_negotiation.cc:206] Beginning negotiation
0806 18:34:37.851015 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 18:34:37.851364 (+ 349us) server_negotiation.cc:407] Connection header received
0806 18:34:37.851569 (+ 205us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:34:37.851572 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:34:37.851616 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:34:37.851659 (+ 43us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:34:37.852434 (+ 775us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:34:37.852973 (+ 539us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:34:37.853999 (+ 1026us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:34:37.854414 (+ 415us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:34:37.854559 (+ 145us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:34:37.856910 (+ 2351us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:34:37.856914 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:34:37.856917 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 18:34:37.856945 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:34:37.858426 (+ 1481us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:34:37.859070 (+ 644us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:34:37.859073 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:34:37.859074 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:34:37.859207 (+ 133us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:34:37.859609 (+ 402us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:34:37.859616 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:34:37.859619 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:34:37.859911 (+ 292us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:34:37.860024 (+ 113us) server_negotiation.cc:1035] Waiting for connection context
0806 18:34:37.860577 (+ 553us) server_negotiation.cc:299] Negotiation successful
0806 18:34:37.860672 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260806 18:34:37.861721 27142 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:34:37.850821 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.194.62:46737 (local address 127.0.0.1:44404)
0806 18:34:37.851234 (+ 413us) negotiation.cc:107] Waiting for socket to connect
0806 18:34:37.851280 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0806 18:34:37.851449 (+ 169us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:34:37.851811 (+ 362us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:34:37.851814 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:34:37.852113 (+ 299us) client_negotiation.cc:189] Negotiated authn=SASL
0806 18:34:37.852278 (+ 165us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:34:37.852304 (+ 26us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:34:37.853108 (+ 804us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:34:37.853116 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:34:37.853861 (+ 745us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:34:37.853870 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:34:37.854594 (+ 724us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:34:37.854598 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:34:37.854718 (+ 120us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:34:37.855239 (+ 521us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 18:34:37.855254 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0806 18:34:37.856729 (+ 1475us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 18:34:37.858542 (+ 1813us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:34:37.858552 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:34:37.858562 (+ 10us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:34:37.858956 (+ 394us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:34:37.859320 (+ 364us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:34:37.859325 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:34:37.859329 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:34:37.859506 (+ 177us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:34:37.860025 (+ 519us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 18:34:37.860029 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 18:34:37.860310 (+ 281us) client_negotiation.cc:769] Sending connection context
0806 18:34:37.860489 (+ 179us) client_negotiation.cc:240] Negotiation successful
0806 18:34:37.860798 (+ 309us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":227,"thread_start_us":155,"threads_started":1}
I20260806 18:34:37.861723 27141 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:34:37.849257 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37794 (local address 127.19.194.61:39189)
0806 18:34:37.849699 (+ 442us) server_negotiation.cc:206] Beginning negotiation
0806 18:34:37.849717 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0806 18:34:37.849758 (+ 41us) server_negotiation.cc:407] Connection header received
0806 18:34:37.850530 (+ 772us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:34:37.850547 (+ 17us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:34:37.850912 (+ 365us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:34:37.851279 (+ 367us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:34:37.852064 (+ 785us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:34:37.853006 (+ 942us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:34:37.853868 (+ 862us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:34:37.854254 (+ 386us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:34:37.854425 (+ 171us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:34:37.857436 (+ 3011us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:34:37.857442 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:34:37.857454 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 18:34:37.857480 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:34:37.859680 (+ 2200us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:34:37.860177 (+ 497us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:34:37.860183 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:34:37.860184 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:34:37.860276 (+ 92us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:34:37.860552 (+ 276us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:34:37.860555 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:34:37.860556 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:34:37.860860 (+ 304us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:34:37.861030 (+ 170us) server_negotiation.cc:1035] Waiting for connection context
0806 18:34:37.861398 (+ 368us) server_negotiation.cc:299] Negotiation successful
0806 18:34:37.861512 (+ 114us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":311,"thread_start_us":153,"threads_started":1}
I20260806 18:34:37.863274 27138 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.19.194.61" port: 39189 } has no permanent_uuid. Determining permanent_uuid...
Aug 06 18:34:37 dist-test-slave-1xrh krb5kdc[26549](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041277, etypes {rep=17 tkt=17 ses=17}, oryx/127.19.194.61@KRBTEST.COM for oryx/127.19.194.61@KRBTEST.COM
I20260806 18:34:37.870095 27142 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:34:37.863767 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.194.61:39189 (local address 127.0.0.1:37804)
0806 18:34:37.863875 (+ 108us) negotiation.cc:107] Waiting for socket to connect
0806 18:34:37.863888 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0806 18:34:37.863961 (+ 73us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:34:37.864221 (+ 260us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:34:37.864223 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:34:37.864349 (+ 126us) client_negotiation.cc:189] Negotiated authn=SASL
0806 18:34:37.864433 (+ 84us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:34:37.864437 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:34:37.865103 (+ 666us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:34:37.865106 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:34:37.865600 (+ 494us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:34:37.865608 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:34:37.866002 (+ 394us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:34:37.866004 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:34:37.866082 (+ 78us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:34:37.866427 (+ 345us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 18:34:37.866437 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0806 18:34:37.867561 (+ 1124us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 18:34:37.868907 (+ 1346us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:34:37.868909 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:34:37.868910 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:34:37.869093 (+ 183us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:34:37.869320 (+ 227us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:34:37.869322 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:34:37.869323 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:34:37.869392 (+ 69us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:34:37.869694 (+ 302us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 18:34:37.869696 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 18:34:37.869763 (+ 67us) client_negotiation.cc:769] Sending connection context
0806 18:34:37.869839 (+ 76us) client_negotiation.cc:240] Negotiation successful
0806 18:34:37.869934 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":43}
I20260806 18:34:37.870116 27141 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:34:37.863837 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37804 (local address 127.19.194.61:39189)
0806 18:34:37.863928 (+ 91us) server_negotiation.cc:206] Beginning negotiation
0806 18:34:37.863931 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 18:34:37.863942 (+ 11us) server_negotiation.cc:407] Connection header received
0806 18:34:37.864068 (+ 126us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:34:37.864072 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:34:37.864116 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:34:37.864197 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:34:37.864525 (+ 328us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:34:37.864974 (+ 449us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:34:37.865698 (+ 724us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:34:37.865909 (+ 211us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:34:37.865984 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:34:37.867689 (+ 1705us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:34:37.867691 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:34:37.867693 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 18:34:37.867714 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:34:37.868813 (+ 1099us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:34:37.869173 (+ 360us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:34:37.869176 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:34:37.869177 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:34:37.869244 (+ 67us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:34:37.869463 (+ 219us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:34:37.869465 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:34:37.869466 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:34:37.869617 (+ 151us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:34:37.869689 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0806 18:34:37.869906 (+ 217us) server_negotiation.cc:299] Negotiation successful
0806 18:34:37.870004 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":33}
I20260806 18:34:37.872900 27138 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9a38e245bcc34630ba3cc6fb6ca11cb3: Bootstrap starting.
I20260806 18:34:37.875047 27138 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 9a38e245bcc34630ba3cc6fb6ca11cb3: Neither blocks nor log segments found. Creating new log.
I20260806 18:34:37.875783 27138 log.cc:824] T 00000000000000000000000000000000 P 9a38e245bcc34630ba3cc6fb6ca11cb3: Log is configured to *not* fsync() on all Append() calls
I20260806 18:34:37.877506 27138 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9a38e245bcc34630ba3cc6fb6ca11cb3: No bootstrap required, opened a new log
I20260806 18:34:37.880435 27138 raft_consensus.cc:348] T 00000000000000000000000000000000 P 9a38e245bcc34630ba3cc6fb6ca11cb3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a475b841d65e4ef59cb6d53d3e454091" member_type: VOTER last_known_addr { host: "127.19.194.62" port: 46737 } } peers { permanent_uuid: "9a38e245bcc34630ba3cc6fb6ca11cb3" member_type: VOTER last_known_addr { host: "127.19.194.61" port: 39189 } }
I20260806 18:34:37.880646 27138 raft_consensus.cc:374] T 00000000000000000000000000000000 P 9a38e245bcc34630ba3cc6fb6ca11cb3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 18:34:37.880728 27138 raft_consensus.cc:729] T 00000000000000000000000000000000 P 9a38e245bcc34630ba3cc6fb6ca11cb3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9a38e245bcc34630ba3cc6fb6ca11cb3, State: Initialized, Role: FOLLOWER
I20260806 18:34:37.881182 27138 consensus_queue.cc:260] T 00000000000000000000000000000000 P 9a38e245bcc34630ba3cc6fb6ca11cb3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a475b841d65e4ef59cb6d53d3e454091" member_type: VOTER last_known_addr { host: "127.19.194.62" port: 46737 } } peers { permanent_uuid: "9a38e245bcc34630ba3cc6fb6ca11cb3" member_type: VOTER last_known_addr { host: "127.19.194.61" port: 39189 } }
I20260806 18:34:37.881641 27144 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9a38e245bcc34630ba3cc6fb6ca11cb3 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a475b841d65e4ef59cb6d53d3e454091" member_type: VOTER last_known_addr { host: "127.19.194.62" port: 46737 } } peers { permanent_uuid: "9a38e245bcc34630ba3cc6fb6ca11cb3" member_type: VOTER last_known_addr { host: "127.19.194.61" port: 39189 } } }
I20260806 18:34:37.881747 27144 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9a38e245bcc34630ba3cc6fb6ca11cb3 [sys.catalog]: This master's current role is: FOLLOWER
I20260806 18:34:37.882239 27144 raft_consensus.cc:482] T 00000000000000000000000000000000 P 9a38e245bcc34630ba3cc6fb6ca11cb3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 18:34:37.882378 27138 sys_catalog.cc:564] T 00000000000000000000000000000000 P 9a38e245bcc34630ba3cc6fb6ca11cb3 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 18:34:37.882383 27144 raft_consensus.cc:504] T 00000000000000000000000000000000 P 9a38e245bcc34630ba3cc6fb6ca11cb3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a475b841d65e4ef59cb6d53d3e454091" member_type: VOTER last_known_addr { host: "127.19.194.62" port: 46737 } } peers { permanent_uuid: "9a38e245bcc34630ba3cc6fb6ca11cb3" member_type: VOTER last_known_addr { host: "127.19.194.61" port: 39189 } }
I20260806 18:34:37.883266 27144 leader_election.cc:288] T 00000000000000000000000000000000 P 9a38e245bcc34630ba3cc6fb6ca11cb3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a475b841d65e4ef59cb6d53d3e454091 (127.19.194.62:46737)
I20260806 18:34:37.883886 26607 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "9a38e245bcc34630ba3cc6fb6ca11cb3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a475b841d65e4ef59cb6d53d3e454091" is_pre_election: true
I20260806 18:34:37.884546 27089 leader_election.cc:302] T 00000000000000000000000000000000 P 9a38e245bcc34630ba3cc6fb6ca11cb3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 2 responses out of 2 voters: 1 yes votes; 1 no votes. yes voters: 9a38e245bcc34630ba3cc6fb6ca11cb3; no voters: a475b841d65e4ef59cb6d53d3e454091
I20260806 18:34:37.885129 27144 raft_consensus.cc:2726] T 00000000000000000000000000000000 P 9a38e245bcc34630ba3cc6fb6ca11cb3 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260806 18:34:37.889629 27070 master_runner.cc:428] Detected that this master 9a38e245bcc34630ba3cc6fb6ca11cb3 is joining an existing cluster
I20260806 18:34:37.889684 27070 master_runner.cc:432] Initiating AddMaster RPC to add 127.19.194.61:39189
W20260806 18:34:37.891949 27155 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 9a38e245bcc34630ba3cc6fb6ca11cb3: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260806 18:34:37.892028 27155 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260806 18:34:37.899081 27162 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:34:37.891688 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.194.61:39189 (local address 127.0.0.1:37814)
0806 18:34:37.892304 (+ 616us) negotiation.cc:107] Waiting for socket to connect
0806 18:34:37.892316 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0806 18:34:37.892435 (+ 119us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:34:37.892675 (+ 240us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:34:37.892677 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:34:37.892869 (+ 192us) client_negotiation.cc:189] Negotiated authn=SASL
0806 18:34:37.892963 (+ 94us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:34:37.892969 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:34:37.893659 (+ 690us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:34:37.893662 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:34:37.894215 (+ 553us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:34:37.894222 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:34:37.894663 (+ 441us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:34:37.894666 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:34:37.894744 (+ 78us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:34:37.895239 (+ 495us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 18:34:37.895252 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0806 18:34:37.895574 (+ 322us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 18:34:37.897526 (+ 1952us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:34:37.897528 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:34:37.897530 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:34:37.897733 (+ 203us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:34:37.898000 (+ 267us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:34:37.898003 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:34:37.898004 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:34:37.898089 (+ 85us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:34:37.898590 (+ 501us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 18:34:37.898594 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 18:34:37.898691 (+ 97us) client_negotiation.cc:769] Sending connection context
0806 18:34:37.898784 (+ 93us) client_negotiation.cc:240] Negotiation successful
0806 18:34:37.898911 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":529,"thread_start_us":94,"threads_started":1}
I20260806 18:34:37.899672 27141 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:34:37.891735 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37814 (local address 127.19.194.61:39189)
0806 18:34:37.891840 (+ 105us) server_negotiation.cc:206] Beginning negotiation
0806 18:34:37.891845 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 18:34:37.892393 (+ 548us) server_negotiation.cc:407] Connection header received
0806 18:34:37.892507 (+ 114us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:34:37.892513 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:34:37.892549 (+ 36us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:34:37.892639 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:34:37.893072 (+ 433us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:34:37.893547 (+ 475us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:34:37.894323 (+ 776us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:34:37.894539 (+ 216us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:34:37.894651 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:34:37.895707 (+ 1056us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:34:37.895710 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:34:37.895713 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 18:34:37.895737 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:34:37.897422 (+ 1685us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:34:37.897824 (+ 402us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:34:37.897829 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:34:37.897830 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:34:37.897916 (+ 86us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:34:37.898174 (+ 258us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:34:37.898179 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:34:37.898181 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:34:37.898445 (+ 264us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:34:37.898554 (+ 109us) server_negotiation.cc:1035] Waiting for connection context
0806 18:34:37.899288 (+ 734us) server_negotiation.cc:299] Negotiation successful
0806 18:34:37.899542 (+ 254us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260806 18:34:37.904132 27161 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:34:37.891985 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.194.62:46737 (local address 127.0.0.1:44406)
0806 18:34:37.895824 (+ 3839us) negotiation.cc:107] Waiting for socket to connect
0806 18:34:37.895839 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0806 18:34:37.895984 (+ 145us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:34:37.896401 (+ 417us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:34:37.896404 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:34:37.896631 (+ 227us) client_negotiation.cc:189] Negotiated authn=SASL
0806 18:34:37.896797 (+ 166us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:34:37.896805 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:34:37.897847 (+ 1042us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:34:37.897849 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:34:37.899409 (+ 1560us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:34:37.899421 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:34:37.900080 (+ 659us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:34:37.900083 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:34:37.900167 (+ 84us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:34:37.900575 (+ 408us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 18:34:37.900589 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0806 18:34:37.900991 (+ 402us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 18:34:37.902652 (+ 1661us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:34:37.902658 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:34:37.902661 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:34:37.902849 (+ 188us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:34:37.903151 (+ 302us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:34:37.903155 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:34:37.903157 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:34:37.903259 (+ 102us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:34:37.903622 (+ 363us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 18:34:37.903625 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 18:34:37.903719 (+ 94us) client_negotiation.cc:769] Sending connection context
0806 18:34:37.903837 (+ 118us) client_negotiation.cc:240] Negotiation successful
0806 18:34:37.903966 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":3773,"thread_start_us":102,"threads_started":1}
I20260806 18:34:37.904150 26636 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:34:37.891728 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44406 (local address 127.19.194.62:46737)
0806 18:34:37.891845 (+ 117us) server_negotiation.cc:206] Beginning negotiation
0806 18:34:37.891850 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 18:34:37.895912 (+ 4062us) server_negotiation.cc:407] Connection header received
0806 18:34:37.896094 (+ 182us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:34:37.896100 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:34:37.896157 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:34:37.896214 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:34:37.896937 (+ 723us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:34:37.897721 (+ 784us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:34:37.899540 (+ 1819us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:34:37.899855 (+ 315us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:34:37.899917 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:34:37.901122 (+ 1205us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:34:37.901126 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:34:37.901129 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 18:34:37.901152 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:34:37.902540 (+ 1388us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:34:37.902946 (+ 406us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:34:37.902950 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:34:37.902952 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:34:37.903031 (+ 79us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:34:37.903352 (+ 321us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:34:37.903357 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:34:37.903358 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:34:37.903523 (+ 165us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:34:37.903631 (+ 108us) server_negotiation.cc:1035] Waiting for connection context
0806 18:34:37.903914 (+ 283us) server_negotiation.cc:299] Negotiation successful
0806 18:34:37.904019 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
Aug 06 18:34:37 dist-test-slave-1xrh krb5kdc[26549](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041276, etypes {rep=17 tkt=17 ses=17}, oryx/127.19.194.62@KRBTEST.COM for oryx/127.19.194.61@KRBTEST.COM
I20260806 18:34:37.916919 27163 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:34:37.907910 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.194.61:39189 (local address 127.0.0.1:37826)
0806 18:34:37.908200 (+ 290us) negotiation.cc:107] Waiting for socket to connect
0806 18:34:37.908234 (+ 34us) client_negotiation.cc:174] Beginning negotiation
0806 18:34:37.908327 (+ 93us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:34:37.908837 (+ 510us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:34:37.908843 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:34:37.909159 (+ 316us) client_negotiation.cc:189] Negotiated authn=SASL
0806 18:34:37.909247 (+ 88us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:34:37.909253 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:34:37.909905 (+ 652us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:34:37.909911 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:34:37.910511 (+ 600us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:34:37.910520 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:34:37.910943 (+ 423us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:34:37.910947 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:34:37.911047 (+ 100us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:34:37.911526 (+ 479us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 18:34:37.911576 (+ 50us) client_negotiation.cc:656] calling sasl_client_start()
0806 18:34:37.913099 (+ 1523us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 18:34:37.914968 (+ 1869us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:34:37.914977 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:34:37.914984 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:34:37.915334 (+ 350us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:34:37.915635 (+ 301us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:34:37.915640 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:34:37.915643 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:34:37.915742 (+ 99us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:34:37.916150 (+ 408us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 18:34:37.916160 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 18:34:37.916298 (+ 138us) client_negotiation.cc:769] Sending connection context
0806 18:34:37.916449 (+ 151us) client_negotiation.cc:240] Negotiation successful
0806 18:34:37.916647 (+ 198us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":155,"thread_start_us":83,"threads_started":1}
I20260806 18:34:37.917441 27141 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:34:37.908491 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37826 (local address 127.19.194.61:39189)
0806 18:34:37.908596 (+ 105us) server_negotiation.cc:206] Beginning negotiation
0806 18:34:37.908599 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 18:34:37.908610 (+ 11us) server_negotiation.cc:407] Connection header received
0806 18:34:37.908656 (+ 46us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:34:37.908658 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:34:37.908692 (+ 34us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:34:37.908798 (+ 106us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:34:37.909370 (+ 572us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:34:37.909795 (+ 425us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:34:37.910622 (+ 827us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:34:37.910829 (+ 207us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:34:37.910940 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:34:37.913250 (+ 2310us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:34:37.913253 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:34:37.913256 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 18:34:37.913283 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:34:37.914813 (+ 1530us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:34:37.915441 (+ 628us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:34:37.915445 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:34:37.915446 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:34:37.915525 (+ 79us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:34:37.915842 (+ 317us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:34:37.915845 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:34:37.915847 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:34:37.916034 (+ 187us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:34:37.916130 (+ 96us) server_negotiation.cc:1035] Waiting for connection context
0806 18:34:37.917128 (+ 998us) server_negotiation.cc:299] Negotiation successful
0806 18:34:37.917251 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260806 18:34:37.918503 26597 catalog_manager.cc:6979] Initiating ChangeConfig request to add master 127.19.194.61:39189: tablet_id: "00000000000000000000000000000000"
type: ADD_PEER
server {
permanent_uuid: "9a38e245bcc34630ba3cc6fb6ca11cb3"
member_type: NON_VOTER
last_known_addr {
host: "127.19.194.61"
port: 39189
}
attrs {
promote: true
}
}
dest_uuid: "a475b841d65e4ef59cb6d53d3e454091"
cas_config_opid_index: -1
I20260806 18:34:37.919091 26597 consensus_queue.cc:237] T 00000000000000000000000000000000 P a475b841d65e4ef59cb6d53d3e454091 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "a475b841d65e4ef59cb6d53d3e454091" member_type: VOTER last_known_addr { host: "127.19.194.62" port: 46737 } } peers { permanent_uuid: "9a38e245bcc34630ba3cc6fb6ca11cb3" member_type: NON_VOTER last_known_addr { host: "127.19.194.61" port: 39189 } attrs { promote: true } }
I20260806 18:34:37.920831 27165 raft_consensus.cc:2932] T 00000000000000000000000000000000 P a475b841d65e4ef59cb6d53d3e454091 [term 1 LEADER]: Committing config change with OpId 1.5: config changed from index -1 to 5, NON_VOTER 9a38e245bcc34630ba3cc6fb6ca11cb3 (127.19.194.61) added. New config: { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "a475b841d65e4ef59cb6d53d3e454091" member_type: VOTER last_known_addr { host: "127.19.194.62" port: 46737 } } peers { permanent_uuid: "9a38e245bcc34630ba3cc6fb6ca11cb3" member_type: NON_VOTER last_known_addr { host: "127.19.194.61" port: 39189 } attrs { promote: true } } }
I20260806 18:34:37.921391 27112 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 9a38e245bcc34630ba3cc6fb6ca11cb3 [term 0 FOLLOWER]: Advancing to term 1
I20260806 18:34:37.922315 27165 catalog_manager.cc:6983] Successfully completed master ChangeConfig request to add master 127.19.194.61:39189
I20260806 18:34:37.922640 27112 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 9a38e245bcc34630ba3cc6fb6ca11cb3 [term 1 FOLLOWER]: Refusing update from remote peer a475b841d65e4ef59cb6d53d3e454091: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20260806 18:34:37.922613 27165 sys_catalog.cc:455] T 00000000000000000000000000000000 P a475b841d65e4ef59cb6d53d3e454091 [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "a475b841d65e4ef59cb6d53d3e454091" committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "a475b841d65e4ef59cb6d53d3e454091" member_type: VOTER last_known_addr { host: "127.19.194.62" port: 46737 } } peers { permanent_uuid: "9a38e245bcc34630ba3cc6fb6ca11cb3" member_type: NON_VOTER last_known_addr { host: "127.19.194.61" port: 39189 } attrs { promote: true } } }
I20260806 18:34:37.922688 27070 master.cc:537] Master@127.19.194.61:39189 shutting down...
I20260806 18:34:37.922704 27165 sys_catalog.cc:458] T 00000000000000000000000000000000 P a475b841d65e4ef59cb6d53d3e454091 [sys.catalog]: This master's current role is: LEADER
WARNING: no policy specified for oryx/127.19.194.60@KRBTEST.COM; defaulting to no policy
I20260806 18:34:37.927840 27165 consensus_queue.cc:1035] T 00000000000000000000000000000000 P a475b841d65e4ef59cb6d53d3e454091 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9a38e245bcc34630ba3cc6fb6ca11cb3" member_type: NON_VOTER last_known_addr { host: "127.19.194.61" port: 39189 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 18:34:37.928254 27070 raft_consensus.cc:2229] T 00000000000000000000000000000000 P 9a38e245bcc34630ba3cc6fb6ca11cb3 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 18:34:37.928519 27070 raft_consensus.cc:2258] T 00000000000000000000000000000000 P 9a38e245bcc34630ba3cc6fb6ca11cb3 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 18:34:37.928632 27070 tablet_replica.cc:330] T 00000000000000000000000000000000 P 9a38e245bcc34630ba3cc6fb6ca11cb3: stopping tablet replica
W20260806 18:34:37.929711 26579 proxy.cc:239] Call had error, refreshing address and retrying: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on Master
W20260806 18:34:37.930668 26579 consensus_peers.cc:487] T 00000000000000000000000000000000 P a475b841d65e4ef59cb6d53d3e454091 -> Peer 9a38e245bcc34630ba3cc6fb6ca11cb3 (127.19.194.61:39189): Couldn't send request to peer 9a38e245bcc34630ba3cc6fb6ca11cb3. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on Master. This is attempt 1: this message will repeat every 5th retry.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.19.194.60@KRBTEST.COM" created.
I20260806 18:34:37.935178 27070 master.cc:552] Master@127.19.194.61:39189 shutdown complete.
I20260806 18:34:37.935228 27070 master_runner.cc:305] Clearing existing system tablet
I20260806 18:34:37.935453 27070 fs_manager.cc:359] Metadata directory not provided
I20260806 18:34:37.935521 27070 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 18:34:37.936720 27070 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.001s
I20260806 18:34:37.937570 27172 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 18:34:37.937789 27070 fs_report.cc:383] FS layout report
--------------------
wal directory:
metadata directory:
1 data directories: /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786041246094603-20232-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 18:34:37.937950 27070 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260806 18:34:37.937989 27070 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786041246094603-20232-0/minicluster-data/master-1/data,/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786041246094603-20232-0/minicluster-data/master-1/wal
uuid: "9a38e245bcc34630ba3cc6fb6ca11cb3"
format_stamp: "Formatted at 2026-08-06 18:34:37 on dist-test-slave-1xrh"
server_key: "4c952ddde25c1a62222a1ee034d444b3"
server_key_iv: "b0791910121edd82c0ccc6dc106e0048"
server_key_version: "encryptionkey@0"
I20260806 18:34:37.939674 27070 ts_tablet_manager.cc:1905] T 00000000000000000000000000000000 P 9a38e245bcc34630ba3cc6fb6ca11cb3: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 18:34:37.942587 27070 ts_tablet_manager.cc:1918] T 00000000000000000000000000000000 P 9a38e245bcc34630ba3cc6fb6ca11cb3: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId unknown
I20260806 18:34:37.942701 27070 log.cc:1196] T 00000000000000000000000000000000 P 9a38e245bcc34630ba3cc6fb6ca11cb3: Deleting WAL directory at /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786041246094603-20232-0/minicluster-data/master-1/wal/wals/00000000000000000000000000000000
I20260806 18:34:37.943101 27070 ts_tablet_manager.cc:1939] T 00000000000000000000000000000000 P 9a38e245bcc34630ba3cc6fb6ca11cb3: Deleting consensus metadata
I20260806 18:34:37.943419 27070 master_runner.cc:315] Copying system tablet from 127.19.194.62:46737
I20260806 18:34:37.944577 27070 tablet_copy_client.cc:316] T 00000000000000000000000000000000 P 9a38e245bcc34630ba3cc6fb6ca11cb3: tablet copy: Beginning tablet copy session from remote peer at address 127.19.194.62:46737
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.19.194.60 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786041246094603-20232-0/krb5kdc/oryx_127.19.194.60.keytab.
Entry for principal oryx/127.19.194.60 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786041246094603-20232-0/krb5kdc/oryx_127.19.194.60.keytab.
I20260806 18:34:37.951098 20232 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJW69tn/build/debug/bin/kudu
/tmp/dist-test-taskJW69tn/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786041246094603-20232-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786041246094603-20232-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786041246094603-20232-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786041246094603-20232-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.19.194.60:41749
--webserver_interface=127.19.194.60
--webserver_port=0
--keytab_file=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786041246094603-20232-0/krb5kdc/oryx_127.19.194.60.keytab
--principal=oryx/127.19.194.60
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.19.194.20:44897
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.19.194.62:46737,127.19.194.61:39189,127.19.194.60:41749
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786041246094603-20232-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786041246094603-20232-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786041246094603-20232-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260806 18:34:37.954279 27178 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:34:37.945059 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.194.62:46737 (local address 127.0.0.1:44414)
0806 18:34:37.945381 (+ 322us) negotiation.cc:107] Waiting for socket to connect
0806 18:34:37.945396 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0806 18:34:37.945515 (+ 119us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:34:37.945888 (+ 373us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:34:37.945892 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:34:37.946111 (+ 219us) client_negotiation.cc:189] Negotiated authn=SASL
0806 18:34:37.946234 (+ 123us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:34:37.946241 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:34:37.947161 (+ 920us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:34:37.947165 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:34:37.948079 (+ 914us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:34:37.948089 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:34:37.948646 (+ 557us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:34:37.948649 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:34:37.948743 (+ 94us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:34:37.949871 (+ 1128us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 18:34:37.949886 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0806 18:34:37.950316 (+ 430us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 18:34:37.952598 (+ 2282us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:34:37.952602 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:34:37.952605 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:34:37.952826 (+ 221us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:34:37.953136 (+ 310us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:34:37.953139 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:34:37.953140 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:34:37.953219 (+ 79us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:34:37.953731 (+ 512us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 18:34:37.953734 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 18:34:37.953838 (+ 104us) client_negotiation.cc:769] Sending connection context
0806 18:34:37.953945 (+ 107us) client_negotiation.cc:240] Negotiation successful
0806 18:34:37.954080 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":229,"thread_start_us":83,"threads_started":1}
I20260806 18:34:37.954407 26636 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:34:37.945188 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44414 (local address 127.19.194.62:46737)
0806 18:34:37.945318 (+ 130us) server_negotiation.cc:206] Beginning negotiation
0806 18:34:37.945324 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 18:34:37.945484 (+ 160us) server_negotiation.cc:407] Connection header received
0806 18:34:37.945604 (+ 120us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:34:37.945607 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:34:37.945648 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:34:37.945754 (+ 106us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:34:37.946366 (+ 612us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:34:37.947038 (+ 672us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:34:37.948266 (+ 1228us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:34:37.948519 (+ 253us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:34:37.948757 (+ 238us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:34:37.950479 (+ 1722us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:34:37.950483 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:34:37.950486 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 18:34:37.950517 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:34:37.952476 (+ 1959us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:34:37.952942 (+ 466us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:34:37.952945 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:34:37.952947 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:34:37.953033 (+ 86us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:34:37.953324 (+ 291us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:34:37.953330 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:34:37.953332 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:34:37.953616 (+ 284us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:34:37.953736 (+ 120us) server_negotiation.cc:1035] Waiting for connection context
0806 18:34:37.954040 (+ 304us) server_negotiation.cc:299] Negotiation successful
0806 18:34:37.954187 (+ 147us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260806 18:34:37.955264 26617 tablet_copy_service.cc:140] P a475b841d65e4ef59cb6d53d3e454091: Received BeginTabletCopySession request for tablet 00000000000000000000000000000000 from peer 9a38e245bcc34630ba3cc6fb6ca11cb3 ({username='oryx', principal='oryx/127.19.194.61@KRBTEST.COM'} at 127.0.0.1:44414)
I20260806 18:34:37.955435 26617 tablet_copy_service.cc:161] P a475b841d65e4ef59cb6d53d3e454091: Beginning new tablet copy session on tablet 00000000000000000000000000000000 from peer 9a38e245bcc34630ba3cc6fb6ca11cb3 at {username='oryx', principal='oryx/127.19.194.61@KRBTEST.COM'} at 127.0.0.1:44414: session id = 9a38e245bcc34630ba3cc6fb6ca11cb3-00000000000000000000000000000000
I20260806 18:34:37.957935 26617 tablet_copy_source_session.cc:205] T 00000000000000000000000000000000 P a475b841d65e4ef59cb6d53d3e454091: Tablet Copy: opened 0 blocks and 1 log segments
I20260806 18:34:37.959797 27070 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total
I20260806 18:34:37.961998 27070 tablet_copy_client.cc:791] T 00000000000000000000000000000000 P 9a38e245bcc34630ba3cc6fb6ca11cb3: tablet copy: Starting download of 0 data blocks...
I20260806 18:34:37.962162 27070 tablet_copy_client.cc:655] T 00000000000000000000000000000000 P 9a38e245bcc34630ba3cc6fb6ca11cb3: tablet copy: Starting download of 1 WAL segments...
I20260806 18:34:37.964176 27070 tablet_copy_client.cc:531] T 00000000000000000000000000000000 P 9a38e245bcc34630ba3cc6fb6ca11cb3: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260806 18:34:37.965924 26617 tablet_copy_service.cc:342] P a475b841d65e4ef59cb6d53d3e454091: Request end of tablet copy session 9a38e245bcc34630ba3cc6fb6ca11cb3-00000000000000000000000000000000 received from {username='oryx', principal='oryx/127.19.194.61@KRBTEST.COM'} at 127.0.0.1:44414
I20260806 18:34:37.966050 26617 tablet_copy_service.cc:433] P a475b841d65e4ef59cb6d53d3e454091: ending tablet copy session 9a38e245bcc34630ba3cc6fb6ca11cb3-00000000000000000000000000000000 on tablet 00000000000000000000000000000000 with peer 9a38e245bcc34630ba3cc6fb6ca11cb3
I20260806 18:34:37.974690 27070 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 18:34:37.976444 27180 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:34:37.976550 27181 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:34:37.976750 27183 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 18:34:37.977624 27070 server_base.cc:1034] running on GCE node
I20260806 18:34:37.977792 27070 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 18:34:37.978083 27070 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 18:34:37.979183 27070 hybrid_clock.cc:648] HybridClock initialized: now 1786041277979156 us; error 47 us; skew 500 ppm
Aug 06 18:34:37 dist-test-slave-1xrh krb5kdc[26549](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041277, etypes {rep=17 tkt=17 ses=17}, oryx/127.19.194.61@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 18:34:37.980927 27070 init.cc:377] Logged in from keytab as oryx/127.19.194.61@KRBTEST.COM (short username oryx)
I20260806 18:34:37.981421 27070 webserver.cc:460] Webserver started at http://127.19.194.61:34997/ using document root <none> and password file <none>
I20260806 18:34:37.981626 27070 fs_manager.cc:359] Metadata directory not provided
I20260806 18:34:37.981673 27070 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 18:34:37.983031 27070 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260806 18:34:37.984151 27190 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 18:34:37.984444 27070 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260806 18:34:37.984524 27070 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786041246094603-20232-0/minicluster-data/master-1/data,/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786041246094603-20232-0/minicluster-data/master-1/wal
uuid: "9a38e245bcc34630ba3cc6fb6ca11cb3"
format_stamp: "Formatted at 2026-08-06 18:34:37 on dist-test-slave-1xrh"
server_key: "4c952ddde25c1a62222a1ee034d444b3"
server_key_iv: "b0791910121edd82c0ccc6dc106e0048"
server_key_version: "encryptionkey@0"
I20260806 18:34:37.984611 27070 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786041246094603-20232-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786041246094603-20232-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786041246094603-20232-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 18:34:37.997395 27070 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 18:34:37.999553 27070 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 18:34:38.004824 27070 rpc_server.cc:307] RPC server started. Bound to: 127.19.194.61:39189
I20260806 18:34:38.005483 27070 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786041246094603-20232-0/minicluster-data/master-1/data/info.pb
I20260806 18:34:38.007484 27243 sys_catalog.cc:263] Verifying existing consensus state
I20260806 18:34:38.008585 27243 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9a38e245bcc34630ba3cc6fb6ca11cb3: Bootstrap starting.
I20260806 18:34:38.010995 27242 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.194.61:39189 every 8 connection(s)
I20260806 18:34:38.019361 27243 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9a38e245bcc34630ba3cc6fb6ca11cb3: Bootstrap replayed 1/1 log segments. Stats: ops{read=5 overwritten=0 applied=5 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260806 18:34:38.019707 27243 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9a38e245bcc34630ba3cc6fb6ca11cb3: Bootstrap complete.
I20260806 18:34:38.020290 27243 raft_consensus.cc:348] T 00000000000000000000000000000000 P 9a38e245bcc34630ba3cc6fb6ca11cb3 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "a475b841d65e4ef59cb6d53d3e454091" member_type: VOTER last_known_addr { host: "127.19.194.62" port: 46737 } } peers { permanent_uuid: "9a38e245bcc34630ba3cc6fb6ca11cb3" member_type: NON_VOTER last_known_addr { host: "127.19.194.61" port: 39189 } attrs { promote: true } }
I20260806 18:34:38.020447 27243 raft_consensus.cc:729] T 00000000000000000000000000000000 P 9a38e245bcc34630ba3cc6fb6ca11cb3 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 9a38e245bcc34630ba3cc6fb6ca11cb3, State: Initialized, Role: LEARNER
I20260806 18:34:38.020565 27243 consensus_queue.cc:260] T 00000000000000000000000000000000 P 9a38e245bcc34630ba3cc6fb6ca11cb3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 5, Last appended: 1.5, Last appended by leader: 5, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "a475b841d65e4ef59cb6d53d3e454091" member_type: VOTER last_known_addr { host: "127.19.194.62" port: 46737 } } peers { permanent_uuid: "9a38e245bcc34630ba3cc6fb6ca11cb3" member_type: NON_VOTER last_known_addr { host: "127.19.194.61" port: 39189 } attrs { promote: true } }
I20260806 18:34:38.021041 27243 sys_catalog.cc:564] T 00000000000000000000000000000000 P 9a38e245bcc34630ba3cc6fb6ca11cb3 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 18:34:38.021026 27246 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9a38e245bcc34630ba3cc6fb6ca11cb3 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "a475b841d65e4ef59cb6d53d3e454091" member_type: VOTER last_known_addr { host: "127.19.194.62" port: 46737 } } peers { permanent_uuid: "9a38e245bcc34630ba3cc6fb6ca11cb3" member_type: NON_VOTER last_known_addr { host: "127.19.194.61" port: 39189 } attrs { promote: true } } }
I20260806 18:34:38.021291 27246 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9a38e245bcc34630ba3cc6fb6ca11cb3 [sys.catalog]: This master's current role is: LEARNER
I20260806 18:34:38.024658 27257 catalog_manager.cc:1259] Loaded cluster ID: 3fbbab1f31df424cb2df0f828bded742
I20260806 18:34:38.024765 27257 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 9a38e245bcc34630ba3cc6fb6ca11cb3: loading cluster ID for follower catalog manager: success
I20260806 18:34:38.027055 27257 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 9a38e245bcc34630ba3cc6fb6ca11cb3: acquiring CA information for follower catalog manager: success
I20260806 18:34:38.028514 27257 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 9a38e245bcc34630ba3cc6fb6ca11cb3: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
W20260806 18:34:38.090224 27179 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 18:34:38.090447 27179 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 18:34:38.090484 27179 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 18:34:38.094429 27179 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 18:34:38.094483 27179 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 18:34:38.094502 27179 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 18:34:38.094550 27179 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 18:34:38.094580 27179 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 18:34:38.099778 27179 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.194.20:44897
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786041246094603-20232-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786041246094603-20232-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.19.194.62:46737,127.19.194.61:39189,127.19.194.60:41749
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786041246094603-20232-0/krb5kdc/oryx_127.19.194.60.keytab
--principal=oryx/127.19.194.60
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.19.194.60:41749
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786041246094603-20232-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.19.194.60
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27179
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786041246094603-20232-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 17:59:13 UTC on ccycloud.quasar-wvsnzx.root.comops.site
build id 4571028
I20260806 18:34:38.100828 27179 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 18:34:38.101828 27179 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 18:34:38.110100 27265 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 18:34:38.110167 27179 server_base.cc:1034] running on GCE node
W20260806 18:34:38.110065 27263 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:34:38.110111 27262 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 18:34:38.110843 27179 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 18:34:38.111907 27179 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 18:34:38.113072 27179 hybrid_clock.cc:648] HybridClock initialized: now 1786041278113056 us; error 26 us; skew 500 ppm
Aug 06 18:34:38 dist-test-slave-1xrh krb5kdc[26549](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041278, etypes {rep=17 tkt=17 ses=17}, oryx/127.19.194.60@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 18:34:38.115952 27179 init.cc:377] Logged in from keytab as oryx/127.19.194.60@KRBTEST.COM (short username oryx)
I20260806 18:34:38.116771 27179 webserver.cc:460] Webserver started at http://127.19.194.60:44387/ using document root <none> and password file <none>
I20260806 18:34:38.117292 27179 fs_manager.cc:359] Metadata directory not provided
I20260806 18:34:38.117381 27179 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 18:34:38.117604 27179 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 18:34:38.119652 27179 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786041246094603-20232-0/minicluster-data/master-2/data/instance:
uuid: "ad824649dcf140339b33a7ec0f8ebdce"
format_stamp: "Formatted at 2026-08-06 18:34:38 on dist-test-slave-1xrh"
server_key: "fb2265fe9667197d32b9a03bb06ca4ae"
server_key_iv: "cf2f54a4fac4e1f3dcca4fb5c530f693"
server_key_version: "encryptionkey@0"
I20260806 18:34:38.120194 27179 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786041246094603-20232-0/minicluster-data/master-2/wal/instance:
uuid: "ad824649dcf140339b33a7ec0f8ebdce"
format_stamp: "Formatted at 2026-08-06 18:34:38 on dist-test-slave-1xrh"
server_key: "fb2265fe9667197d32b9a03bb06ca4ae"
server_key_iv: "cf2f54a4fac4e1f3dcca4fb5c530f693"
server_key_version: "encryptionkey@0"
I20260806 18:34:38.123994 27179 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260806 18:34:38.126204 27272 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 18:34:38.127442 27179 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260806 18:34:38.127584 27179 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786041246094603-20232-0/minicluster-data/master-2/data,/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786041246094603-20232-0/minicluster-data/master-2/wal
uuid: "ad824649dcf140339b33a7ec0f8ebdce"
format_stamp: "Formatted at 2026-08-06 18:34:38 on dist-test-slave-1xrh"
server_key: "fb2265fe9667197d32b9a03bb06ca4ae"
server_key_iv: "cf2f54a4fac4e1f3dcca4fb5c530f693"
server_key_version: "encryptionkey@0"
I20260806 18:34:38.127691 27179 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786041246094603-20232-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786041246094603-20232-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786041246094603-20232-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 18:34:38.139716 27179 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 18:34:38.142791 27179 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 18:34:38.142980 27179 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 18:34:38.149906 27179 rpc_server.cc:307] RPC server started. Bound to: 127.19.194.60:41749
I20260806 18:34:38.150197 27327 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.194.60:41749 every 8 connection(s)
I20260806 18:34:38.150813 27179 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786041246094603-20232-0/minicluster-data/master-2/data/info.pb
I20260806 18:34:38.153416 27328 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 18:34:38.157027 27328 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.19.194.62" port: 46737 } has no permanent_uuid. Determining permanent_uuid...
I20260806 18:34:38.159789 20232 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJW69tn/build/debug/bin/kudu as pid 27179
Aug 06 18:34:38 dist-test-slave-1xrh krb5kdc[26549](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041278, etypes {rep=17 tkt=17 ses=17}, oryx/127.19.194.60@KRBTEST.COM for oryx/127.19.194.62@KRBTEST.COM
Aug 06 18:34:38 dist-test-slave-1xrh krb5kdc[26549](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041276, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.19.194.60@KRBTEST.COM
I20260806 18:34:38.171455 26636 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:34:38.158834 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44430 (local address 127.19.194.62:46737)
0806 18:34:38.158975 (+ 141us) server_negotiation.cc:206] Beginning negotiation
0806 18:34:38.158979 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 18:34:38.159741 (+ 762us) server_negotiation.cc:407] Connection header received
0806 18:34:38.161110 (+ 1369us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:34:38.161114 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:34:38.161175 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:34:38.161240 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:34:38.162504 (+ 1264us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:34:38.163007 (+ 503us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:34:38.164195 (+ 1188us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:34:38.164518 (+ 323us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:34:38.164605 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:34:38.167292 (+ 2687us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:34:38.167296 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:34:38.167299 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 18:34:38.167332 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:34:38.169037 (+ 1705us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:34:38.169694 (+ 657us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:34:38.169697 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:34:38.169699 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:34:38.169787 (+ 88us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:34:38.170159 (+ 372us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:34:38.170162 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:34:38.170163 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:34:38.170427 (+ 264us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:34:38.170507 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0806 18:34:38.171080 (+ 573us) server_negotiation.cc:299] Negotiation successful
0806 18:34:38.171252 (+ 172us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260806 18:34:38.172356 27332 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:34:38.160636 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44152 (local address 127.19.194.60:41749)
0806 18:34:38.161103 (+ 467us) server_negotiation.cc:206] Beginning negotiation
0806 18:34:38.161115 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0806 18:34:38.161149 (+ 34us) server_negotiation.cc:407] Connection header received
0806 18:34:38.161282 (+ 133us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:34:38.161287 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:34:38.161493 (+ 206us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:34:38.161667 (+ 174us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:34:38.162345 (+ 678us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:34:38.163478 (+ 1133us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:34:38.164448 (+ 970us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:34:38.164924 (+ 476us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:34:38.165040 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:34:38.167792 (+ 2752us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:34:38.167798 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:34:38.167814 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 18:34:38.167835 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:34:38.169577 (+ 1742us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:34:38.170048 (+ 471us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:34:38.170053 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:34:38.170055 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:34:38.170138 (+ 83us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:34:38.170413 (+ 275us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:34:38.170415 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:34:38.170416 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:34:38.170843 (+ 427us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:34:38.171034 (+ 191us) server_negotiation.cc:1035] Waiting for connection context
0806 18:34:38.171266 (+ 232us) server_negotiation.cc:299] Negotiation successful
0806 18:34:38.171380 (+ 114us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":93,"server-negotiator.queue_time_us":358,"spinlock_wait_cycles":23680,"thread_start_us":151,"threads_started":1}
I20260806 18:34:38.172358 27330 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:34:38.159077 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.194.62:46737 (local address 127.0.0.1:44430)
0806 18:34:38.159593 (+ 516us) negotiation.cc:107] Waiting for socket to connect
0806 18:34:38.159633 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0806 18:34:38.160825 (+ 1192us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:34:38.161321 (+ 496us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:34:38.161329 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:34:38.161760 (+ 431us) client_negotiation.cc:189] Negotiated authn=SASL
0806 18:34:38.162355 (+ 595us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:34:38.162368 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:34:38.163181 (+ 813us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:34:38.163184 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:34:38.164084 (+ 900us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:34:38.164094 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:34:38.164655 (+ 561us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:34:38.164661 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:34:38.164854 (+ 193us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:34:38.165463 (+ 609us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 18:34:38.165479 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0806 18:34:38.167097 (+ 1618us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 18:34:38.169152 (+ 2055us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:34:38.169159 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:34:38.169168 (+ 9us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:34:38.169575 (+ 407us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:34:38.169882 (+ 307us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:34:38.169884 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:34:38.169885 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:34:38.170052 (+ 167us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:34:38.170530 (+ 478us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 18:34:38.170536 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 18:34:38.170843 (+ 307us) client_negotiation.cc:769] Sending connection context
0806 18:34:38.170996 (+ 153us) client_negotiation.cc:240] Negotiation successful
0806 18:34:38.171313 (+ 317us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":323,"mutex_wait_us":54,"spinlock_wait_cycles":6400,"thread_start_us":114,"threads_started":1}
I20260806 18:34:38.175653 27328 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.19.194.61" port: 39189 } has no permanent_uuid. Determining permanent_uuid...
Aug 06 18:34:38 dist-test-slave-1xrh krb5kdc[26549](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041278, etypes {rep=17 tkt=17 ses=17}, oryx/127.19.194.60@KRBTEST.COM for oryx/127.19.194.61@KRBTEST.COM
I20260806 18:34:38.183269 27330 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:34:38.176090 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.194.61:39189 (local address 127.0.0.1:37834)
0806 18:34:38.176201 (+ 111us) negotiation.cc:107] Waiting for socket to connect
0806 18:34:38.176216 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0806 18:34:38.176285 (+ 69us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:34:38.176696 (+ 411us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:34:38.176699 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:34:38.176859 (+ 160us) client_negotiation.cc:189] Negotiated authn=SASL
0806 18:34:38.176939 (+ 80us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:34:38.176945 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:34:38.177845 (+ 900us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:34:38.177850 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:34:38.178377 (+ 527us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:34:38.178385 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:34:38.178871 (+ 486us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:34:38.178875 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:34:38.178970 (+ 95us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:34:38.179389 (+ 419us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 18:34:38.179400 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0806 18:34:38.180507 (+ 1107us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 18:34:38.182045 (+ 1538us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:34:38.182048 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:34:38.182049 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:34:38.182231 (+ 182us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:34:38.182468 (+ 237us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:34:38.182470 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:34:38.182471 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:34:38.182544 (+ 73us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:34:38.182855 (+ 311us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 18:34:38.182857 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 18:34:38.182930 (+ 73us) client_negotiation.cc:769] Sending connection context
0806 18:34:38.183011 (+ 81us) client_negotiation.cc:240] Negotiation successful
0806 18:34:38.183110 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":46}
I20260806 18:34:38.183372 27333 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:34:38.176227 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37834 (local address 127.19.194.61:39189)
0806 18:34:38.176490 (+ 263us) server_negotiation.cc:206] Beginning negotiation
0806 18:34:38.176493 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 18:34:38.176505 (+ 12us) server_negotiation.cc:407] Connection header received
0806 18:34:38.176540 (+ 35us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:34:38.176542 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:34:38.176581 (+ 39us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:34:38.176682 (+ 101us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:34:38.177060 (+ 378us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:34:38.177731 (+ 671us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:34:38.178520 (+ 789us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:34:38.178756 (+ 236us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:34:38.178872 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:34:38.180637 (+ 1765us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:34:38.180640 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:34:38.180642 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 18:34:38.180663 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:34:38.181928 (+ 1265us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:34:38.182314 (+ 386us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:34:38.182316 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:34:38.182318 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:34:38.182389 (+ 71us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:34:38.182620 (+ 231us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:34:38.182622 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:34:38.182623 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:34:38.182778 (+ 155us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:34:38.182850 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0806 18:34:38.183067 (+ 217us) server_negotiation.cc:299] Negotiation successful
0806 18:34:38.183216 (+ 149us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":193,"thread_start_us":83,"threads_started":1}
I20260806 18:34:38.184046 27328 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.19.194.60" port: 41749 } has no permanent_uuid. Determining permanent_uuid...
Aug 06 18:34:38 dist-test-slave-1xrh krb5kdc[26549](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041278, etypes {rep=17 tkt=17 ses=17}, oryx/127.19.194.60@KRBTEST.COM for oryx/127.19.194.60@KRBTEST.COM
I20260806 18:34:38.190980 27330 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:34:38.184512 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.194.60:41749 (local address 127.0.0.1:44160)
0806 18:34:38.184629 (+ 117us) negotiation.cc:107] Waiting for socket to connect
0806 18:34:38.184644 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0806 18:34:38.184712 (+ 68us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:34:38.184936 (+ 224us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:34:38.184938 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:34:38.185157 (+ 219us) client_negotiation.cc:189] Negotiated authn=SASL
0806 18:34:38.185228 (+ 71us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:34:38.185233 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:34:38.185909 (+ 676us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:34:38.185912 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:34:38.186388 (+ 476us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:34:38.186396 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:34:38.186794 (+ 398us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:34:38.186796 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:34:38.186853 (+ 57us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:34:38.187304 (+ 451us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 18:34:38.187316 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0806 18:34:38.188391 (+ 1075us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 18:34:38.189817 (+ 1426us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:34:38.189819 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:34:38.189821 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:34:38.189996 (+ 175us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:34:38.190231 (+ 235us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:34:38.190233 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:34:38.190234 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:34:38.190322 (+ 88us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:34:38.190622 (+ 300us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 18:34:38.190624 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 18:34:38.190694 (+ 70us) client_negotiation.cc:769] Sending connection context
0806 18:34:38.190770 (+ 76us) client_negotiation.cc:240] Negotiation successful
0806 18:34:38.190857 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":54}
I20260806 18:34:38.191090 27332 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:34:38.184552 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44160 (local address 127.19.194.60:41749)
0806 18:34:38.184661 (+ 109us) server_negotiation.cc:206] Beginning negotiation
0806 18:34:38.184665 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 18:34:38.184681 (+ 16us) server_negotiation.cc:407] Connection header received
0806 18:34:38.184806 (+ 125us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:34:38.184808 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:34:38.184843 (+ 35us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:34:38.184906 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:34:38.185329 (+ 423us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:34:38.185812 (+ 483us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:34:38.186485 (+ 673us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:34:38.186703 (+ 218us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:34:38.186775 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:34:38.188535 (+ 1760us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:34:38.188538 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:34:38.188540 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 18:34:38.188560 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:34:38.189719 (+ 1159us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:34:38.190085 (+ 366us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:34:38.190088 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:34:38.190089 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:34:38.190155 (+ 66us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:34:38.190396 (+ 241us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:34:38.190398 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:34:38.190399 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:34:38.190544 (+ 145us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:34:38.190614 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0806 18:34:38.190824 (+ 210us) server_negotiation.cc:299] Negotiation successful
0806 18:34:38.190909 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260806 18:34:38.194041 27328 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ad824649dcf140339b33a7ec0f8ebdce: Bootstrap starting.
I20260806 18:34:38.196334 27328 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P ad824649dcf140339b33a7ec0f8ebdce: Neither blocks nor log segments found. Creating new log.
I20260806 18:34:38.197008 27328 log.cc:824] T 00000000000000000000000000000000 P ad824649dcf140339b33a7ec0f8ebdce: Log is configured to *not* fsync() on all Append() calls
I20260806 18:34:38.197184 26790 heartbeater.cc:503] Master 127.19.194.62:46737 was elected leader, sending a full tablet report...
I20260806 18:34:38.199031 27328 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ad824649dcf140339b33a7ec0f8ebdce: No bootstrap required, opened a new log
I20260806 18:34:38.202013 27328 raft_consensus.cc:348] T 00000000000000000000000000000000 P ad824649dcf140339b33a7ec0f8ebdce [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a475b841d65e4ef59cb6d53d3e454091" member_type: VOTER last_known_addr { host: "127.19.194.62" port: 46737 } } peers { permanent_uuid: "9a38e245bcc34630ba3cc6fb6ca11cb3" member_type: VOTER last_known_addr { host: "127.19.194.61" port: 39189 } } peers { permanent_uuid: "ad824649dcf140339b33a7ec0f8ebdce" member_type: VOTER last_known_addr { host: "127.19.194.60" port: 41749 } }
I20260806 18:34:38.202194 27328 raft_consensus.cc:374] T 00000000000000000000000000000000 P ad824649dcf140339b33a7ec0f8ebdce [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 18:34:38.202234 27328 raft_consensus.cc:729] T 00000000000000000000000000000000 P ad824649dcf140339b33a7ec0f8ebdce [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ad824649dcf140339b33a7ec0f8ebdce, State: Initialized, Role: FOLLOWER
I20260806 18:34:38.202587 27328 consensus_queue.cc:260] T 00000000000000000000000000000000 P ad824649dcf140339b33a7ec0f8ebdce [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a475b841d65e4ef59cb6d53d3e454091" member_type: VOTER last_known_addr { host: "127.19.194.62" port: 46737 } } peers { permanent_uuid: "9a38e245bcc34630ba3cc6fb6ca11cb3" member_type: VOTER last_known_addr { host: "127.19.194.61" port: 39189 } } peers { permanent_uuid: "ad824649dcf140339b33a7ec0f8ebdce" member_type: VOTER last_known_addr { host: "127.19.194.60" port: 41749 } }
I20260806 18:34:38.202975 27335 sys_catalog.cc:455] T 00000000000000000000000000000000 P ad824649dcf140339b33a7ec0f8ebdce [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a475b841d65e4ef59cb6d53d3e454091" member_type: VOTER last_known_addr { host: "127.19.194.62" port: 46737 } } peers { permanent_uuid: "9a38e245bcc34630ba3cc6fb6ca11cb3" member_type: VOTER last_known_addr { host: "127.19.194.61" port: 39189 } } peers { permanent_uuid: "ad824649dcf140339b33a7ec0f8ebdce" member_type: VOTER last_known_addr { host: "127.19.194.60" port: 41749 } } }
I20260806 18:34:38.203122 27335 sys_catalog.cc:458] T 00000000000000000000000000000000 P ad824649dcf140339b33a7ec0f8ebdce [sys.catalog]: This master's current role is: FOLLOWER
I20260806 18:34:38.203683 27328 sys_catalog.cc:564] T 00000000000000000000000000000000 P ad824649dcf140339b33a7ec0f8ebdce [sys.catalog]: configured and running, proceeding with master startup.
W20260806 18:34:38.208653 27346 catalog_manager.cc:1558] T 00000000000000000000000000000000 P ad824649dcf140339b33a7ec0f8ebdce: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260806 18:34:38.208735 27346 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260806 18:34:38.213045 27179 master_runner.cc:428] Detected that this master ad824649dcf140339b33a7ec0f8ebdce is joining an existing cluster
I20260806 18:34:38.213136 27179 master_runner.cc:432] Initiating AddMaster RPC to add 127.19.194.60:41749
I20260806 18:34:38.223851 27353 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:34:38.215674 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.194.60:41749 (local address 127.0.0.1:44166)
0806 18:34:38.215946 (+ 272us) negotiation.cc:107] Waiting for socket to connect
0806 18:34:38.215960 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0806 18:34:38.216047 (+ 87us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:34:38.216284 (+ 237us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:34:38.216286 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:34:38.216519 (+ 233us) client_negotiation.cc:189] Negotiated authn=SASL
0806 18:34:38.216641 (+ 122us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:34:38.216649 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:34:38.217413 (+ 764us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:34:38.217415 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:34:38.217977 (+ 562us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:34:38.217983 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:34:38.218679 (+ 696us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:34:38.218682 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:34:38.218778 (+ 96us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:34:38.219925 (+ 1147us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 18:34:38.219943 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0806 18:34:38.220487 (+ 544us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 18:34:38.222296 (+ 1809us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:34:38.222302 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:34:38.222305 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:34:38.222529 (+ 224us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:34:38.222867 (+ 338us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:34:38.222869 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:34:38.222870 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:34:38.222951 (+ 81us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:34:38.223385 (+ 434us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 18:34:38.223404 (+ 19us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 18:34:38.223525 (+ 121us) client_negotiation.cc:769] Sending connection context
0806 18:34:38.223612 (+ 87us) client_negotiation.cc:240] Negotiation successful
0806 18:34:38.223706 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":189,"thread_start_us":81,"threads_started":1}
I20260806 18:34:38.223973 27332 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:34:38.215996 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44166 (local address 127.19.194.60:41749)
0806 18:34:38.216105 (+ 109us) server_negotiation.cc:206] Beginning negotiation
0806 18:34:38.216110 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 18:34:38.216121 (+ 11us) server_negotiation.cc:407] Connection header received
0806 18:34:38.216148 (+ 27us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:34:38.216150 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:34:38.216182 (+ 32us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:34:38.216271 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:34:38.216766 (+ 495us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:34:38.217320 (+ 554us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:34:38.218089 (+ 769us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:34:38.218460 (+ 371us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:34:38.218557 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:34:38.220613 (+ 2056us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:34:38.220616 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:34:38.220619 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 18:34:38.220640 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:34:38.222185 (+ 1545us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:34:38.222667 (+ 482us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:34:38.222671 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:34:38.222673 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:34:38.222768 (+ 95us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:34:38.223047 (+ 279us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:34:38.223051 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:34:38.223052 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:34:38.223292 (+ 240us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:34:38.223447 (+ 155us) server_negotiation.cc:1035] Waiting for connection context
0806 18:34:38.223671 (+ 224us) server_negotiation.cc:299] Negotiation successful
0806 18:34:38.223794 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":32,"server-negotiator.queue_time_us":41}
I20260806 18:34:38.225826 20232 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJW69tn/build/debug/bin/kudu with pid 26659
I20260806 18:34:38.236053 20232 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJW69tn/build/debug/bin/kudu
/tmp/dist-test-taskJW69tn/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786041246094603-20232-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786041246094603-20232-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786041246094603-20232-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786041246094603-20232-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.19.194.1:33871
--local_ip_for_outbound_sockets=127.19.194.1
--tserver_master_addrs=127.19.194.62:46737
--webserver_port=33093
--webserver_interface=127.19.194.1
--keytab_file=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786041246094603-20232-0/krb5kdc/oryx_127.19.194.1.keytab
--principal=oryx/127.19.194.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.19.194.20:44897
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.19.194.62:46737,127.19.194.61:39189
--tserver_master_addrs=127.19.194.62:46737,127.19.194.61:39189,127.19.194.60:41749 with env {KRB5CCNAME=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786041246094603-20232-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786041246094603-20232-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786041246094603-20232-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260806 18:34:38.248226 27333 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:34:38.215309 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37848 (local address 127.19.194.61:39189)
0806 18:34:38.215456 (+ 147us) server_negotiation.cc:206] Beginning negotiation
0806 18:34:38.215461 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 18:34:38.215626 (+ 165us) server_negotiation.cc:407] Connection header received
0806 18:34:38.215747 (+ 121us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:34:38.215750 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:34:38.215798 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:34:38.215884 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:34:38.216592 (+ 708us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:34:38.217422 (+ 830us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:34:38.218721 (+ 1299us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:34:38.219454 (+ 733us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:34:38.219543 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:34:38.244534 (+ 24991us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:34:38.244539 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:34:38.244542 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 18:34:38.244574 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:34:38.246319 (+ 1745us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:34:38.246824 (+ 505us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:34:38.246828 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:34:38.246829 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:34:38.246910 (+ 81us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:34:38.247253 (+ 343us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:34:38.247257 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:34:38.247259 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:34:38.247490 (+ 231us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:34:38.247573 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0806 18:34:38.247920 (+ 347us) server_negotiation.cc:299] Negotiation successful
0806 18:34:38.248058 (+ 138us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260806 18:34:38.248783 27352 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:34:38.215264 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.194.61:39189 (local address 127.0.0.1:37848)
0806 18:34:38.215529 (+ 265us) negotiation.cc:107] Waiting for socket to connect
0806 18:34:38.215546 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0806 18:34:38.215644 (+ 98us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:34:38.216012 (+ 368us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:34:38.216015 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:34:38.216243 (+ 228us) client_negotiation.cc:189] Negotiated authn=SASL
0806 18:34:38.216433 (+ 190us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:34:38.216442 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:34:38.217581 (+ 1139us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:34:38.217584 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:34:38.218577 (+ 993us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:34:38.218590 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:34:38.219693 (+ 1103us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:34:38.219696 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:34:38.219805 (+ 109us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:34:38.243770 (+ 23965us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 18:34:38.243799 (+ 29us) client_negotiation.cc:656] calling sasl_client_start()
0806 18:34:38.244317 (+ 518us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 18:34:38.246454 (+ 2137us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:34:38.246457 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:34:38.246459 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:34:38.246707 (+ 248us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:34:38.247020 (+ 313us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:34:38.247023 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:34:38.247025 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:34:38.247114 (+ 89us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:34:38.247650 (+ 536us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 18:34:38.247653 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 18:34:38.247748 (+ 95us) client_negotiation.cc:769] Sending connection context
0806 18:34:38.248437 (+ 689us) client_negotiation.cc:240] Negotiation successful
0806 18:34:38.248603 (+ 166us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":165,"mutex_wait_us":50,"thread_start_us":92,"threads_started":1}
I20260806 18:34:38.249082 26636 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:34:38.216562 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44436 (local address 127.19.194.62:46737)
0806 18:34:38.216686 (+ 124us) server_negotiation.cc:206] Beginning negotiation
0806 18:34:38.216692 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 18:34:38.218292 (+ 1600us) server_negotiation.cc:407] Connection header received
0806 18:34:38.218454 (+ 162us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:34:38.218457 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:34:38.218524 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:34:38.218573 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:34:38.219204 (+ 631us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:34:38.219704 (+ 500us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:34:38.220984 (+ 1280us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:34:38.221381 (+ 397us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:34:38.221484 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:34:38.244732 (+ 23248us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:34:38.244740 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:34:38.244746 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 18:34:38.244783 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:34:38.247227 (+ 2444us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:34:38.247741 (+ 514us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:34:38.247745 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:34:38.247747 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:34:38.247827 (+ 80us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:34:38.248121 (+ 294us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:34:38.248124 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:34:38.248126 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:34:38.248353 (+ 227us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:34:38.248423 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0806 18:34:38.248791 (+ 368us) server_negotiation.cc:299] Negotiation successful
0806 18:34:38.248930 (+ 139us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260806 18:34:38.250077 27354 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:34:38.216705 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.194.62:46737 (local address 127.0.0.1:44436)
0806 18:34:38.218220 (+ 1515us) negotiation.cc:107] Waiting for socket to connect
0806 18:34:38.218231 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0806 18:34:38.218349 (+ 118us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:34:38.218714 (+ 365us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:34:38.218716 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:34:38.218944 (+ 228us) client_negotiation.cc:189] Negotiated authn=SASL
0806 18:34:38.219059 (+ 115us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:34:38.219065 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:34:38.219838 (+ 773us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:34:38.219844 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:34:38.220863 (+ 1019us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:34:38.220873 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:34:38.221492 (+ 619us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:34:38.221496 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:34:38.221640 (+ 144us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:34:38.243935 (+ 22295us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 18:34:38.243953 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0806 18:34:38.244522 (+ 569us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 18:34:38.247366 (+ 2844us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:34:38.247369 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:34:38.247371 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:34:38.247625 (+ 254us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:34:38.247935 (+ 310us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:34:38.247940 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:34:38.247943 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:34:38.248026 (+ 83us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:34:38.248511 (+ 485us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 18:34:38.248513 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 18:34:38.248611 (+ 98us) client_negotiation.cc:769] Sending connection context
0806 18:34:38.249578 (+ 967us) client_negotiation.cc:240] Negotiation successful
0806 18:34:38.249721 (+ 143us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1445,"thread_start_us":91,"threads_started":1}
Aug 06 18:34:38 dist-test-slave-1xrh krb5kdc[26549](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041276, etypes {rep=17 tkt=17 ses=17}, oryx/127.19.194.62@KRBTEST.COM for oryx/127.19.194.60@KRBTEST.COM
I20260806 18:34:38.263250 27332 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:34:38.253978 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44174 (local address 127.19.194.60:41749)
0806 18:34:38.254319 (+ 341us) server_negotiation.cc:206] Beginning negotiation
0806 18:34:38.254323 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 18:34:38.254334 (+ 11us) server_negotiation.cc:407] Connection header received
0806 18:34:38.254396 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:34:38.254399 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:34:38.254447 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:34:38.254531 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:34:38.254984 (+ 453us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:34:38.255853 (+ 869us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:34:38.257014 (+ 1161us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:34:38.257390 (+ 376us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:34:38.257493 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:34:38.259761 (+ 2268us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:34:38.259765 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:34:38.259768 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 18:34:38.259795 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:34:38.261522 (+ 1727us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:34:38.261988 (+ 466us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:34:38.261992 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:34:38.261993 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:34:38.262078 (+ 85us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:34:38.262426 (+ 348us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:34:38.262428 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:34:38.262430 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:34:38.262621 (+ 191us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:34:38.262713 (+ 92us) server_negotiation.cc:1035] Waiting for connection context
0806 18:34:38.262985 (+ 272us) server_negotiation.cc:299] Negotiation successful
0806 18:34:38.263088 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":278}
I20260806 18:34:38.263295 27163 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:34:38.253902 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.194.60:41749 (local address 127.0.0.1:44174)
0806 18:34:38.254098 (+ 196us) negotiation.cc:107] Waiting for socket to connect
0806 18:34:38.254112 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0806 18:34:38.254187 (+ 75us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:34:38.254578 (+ 391us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:34:38.254581 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:34:38.254761 (+ 180us) client_negotiation.cc:189] Negotiated authn=SASL
0806 18:34:38.254859 (+ 98us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:34:38.254865 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:34:38.256010 (+ 1145us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:34:38.256014 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:34:38.256887 (+ 873us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:34:38.256897 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:34:38.257513 (+ 616us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:34:38.257520 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:34:38.257640 (+ 120us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:34:38.258059 (+ 419us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 18:34:38.258072 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0806 18:34:38.259605 (+ 1533us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 18:34:38.261636 (+ 2031us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:34:38.261639 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:34:38.261641 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:34:38.261882 (+ 241us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:34:38.262180 (+ 298us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:34:38.262183 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:34:38.262185 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:34:38.262277 (+ 92us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:34:38.262722 (+ 445us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 18:34:38.262725 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 18:34:38.262818 (+ 93us) client_negotiation.cc:769] Sending connection context
0806 18:34:38.262923 (+ 105us) client_negotiation.cc:240] Negotiation successful
0806 18:34:38.263052 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":122}
I20260806 18:34:38.264696 26597 catalog_manager.cc:6979] Initiating ChangeConfig request to add master 127.19.194.60:41749: tablet_id: "00000000000000000000000000000000"
type: ADD_PEER
server {
permanent_uuid: "ad824649dcf140339b33a7ec0f8ebdce"
member_type: NON_VOTER
last_known_addr {
host: "127.19.194.60"
port: 41749
}
attrs {
promote: true
}
}
dest_uuid: "a475b841d65e4ef59cb6d53d3e454091"
cas_config_opid_index: 5
I20260806 18:34:38.265304 26597 consensus_queue.cc:237] T 00000000000000000000000000000000 P a475b841d65e4ef59cb6d53d3e454091 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 1.5, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "a475b841d65e4ef59cb6d53d3e454091" member_type: VOTER last_known_addr { host: "127.19.194.62" port: 46737 } } peers { permanent_uuid: "9a38e245bcc34630ba3cc6fb6ca11cb3" member_type: NON_VOTER last_known_addr { host: "127.19.194.61" port: 39189 } attrs { promote: true } } peers { permanent_uuid: "ad824649dcf140339b33a7ec0f8ebdce" member_type: NON_VOTER last_known_addr { host: "127.19.194.60" port: 41749 } attrs { promote: true } }
I20260806 18:34:38.266366 27166 raft_consensus.cc:2932] T 00000000000000000000000000000000 P a475b841d65e4ef59cb6d53d3e454091 [term 1 LEADER]: Committing config change with OpId 1.6: config changed from index 5 to 6, NON_VOTER ad824649dcf140339b33a7ec0f8ebdce (127.19.194.60) added. New config: { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "a475b841d65e4ef59cb6d53d3e454091" member_type: VOTER last_known_addr { host: "127.19.194.62" port: 46737 } } peers { permanent_uuid: "9a38e245bcc34630ba3cc6fb6ca11cb3" member_type: NON_VOTER last_known_addr { host: "127.19.194.61" port: 39189 } attrs { promote: true } } peers { permanent_uuid: "ad824649dcf140339b33a7ec0f8ebdce" member_type: NON_VOTER last_known_addr { host: "127.19.194.60" port: 41749 } attrs { promote: true } } }
I20260806 18:34:38.267951 27166 catalog_manager.cc:6983] Successfully completed master ChangeConfig request to add master 127.19.194.60:41749
I20260806 18:34:38.268111 27166 sys_catalog.cc:455] T 00000000000000000000000000000000 P a475b841d65e4ef59cb6d53d3e454091 [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "a475b841d65e4ef59cb6d53d3e454091" committed_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "a475b841d65e4ef59cb6d53d3e454091" member_type: VOTER last_known_addr { host: "127.19.194.62" port: 46737 } } peers { permanent_uuid: "9a38e245bcc34630ba3cc6fb6ca11cb3" member_type: NON_VOTER last_known_addr { host: "127.19.194.61" port: 39189 } attrs { promote: true } } peers { permanent_uuid: "ad824649dcf140339b33a7ec0f8ebdce" member_type: NON_VOTER last_known_addr { host: "127.19.194.60" port: 41749 } attrs { promote: true } } }
I20260806 18:34:38.268225 27166 sys_catalog.cc:458] T 00000000000000000000000000000000 P a475b841d65e4ef59cb6d53d3e454091 [sys.catalog]: This master's current role is: LEADER
I20260806 18:34:38.269002 27301 raft_consensus.cc:3037] T 00000000000000000000000000000000 P ad824649dcf140339b33a7ec0f8ebdce [term 0 FOLLOWER]: Advancing to term 1
I20260806 18:34:38.269151 27179 master.cc:537] Master@127.19.194.60:41749 shutting down...
I20260806 18:34:38.270655 27301 raft_consensus.cc:1261] T 00000000000000000000000000000000 P ad824649dcf140339b33a7ec0f8ebdce [term 1 FOLLOWER]: Refusing update from remote peer a475b841d65e4ef59cb6d53d3e454091: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20260806 18:34:38.271552 27165 consensus_queue.cc:1035] T 00000000000000000000000000000000 P a475b841d65e4ef59cb6d53d3e454091 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ad824649dcf140339b33a7ec0f8ebdce" member_type: NON_VOTER last_known_addr { host: "127.19.194.60" port: 41749 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 0, Time since last communication: 0.000s
W20260806 18:34:38.273244 26579 consensus_peers.cc:487] T 00000000000000000000000000000000 P a475b841d65e4ef59cb6d53d3e454091 -> Peer ad824649dcf140339b33a7ec0f8ebdce (127.19.194.60:41749): Couldn't send request to peer ad824649dcf140339b33a7ec0f8ebdce. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on Master. This is attempt 1: this message will repeat every 5th retry.
I20260806 18:34:38.274277 27163 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:34:38.268619 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.194.61:39189 (local address 127.0.0.1:37854)
0806 18:34:38.268726 (+ 107us) negotiation.cc:107] Waiting for socket to connect
0806 18:34:38.268740 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0806 18:34:38.268814 (+ 74us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:34:38.269536 (+ 722us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:34:38.269540 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:34:38.269566 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 18:34:38.269731 (+ 165us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:34:38.269740 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:34:38.271238 (+ 1498us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:34:38.271241 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:34:38.272657 (+ 1416us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:34:38.272669 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:34:38.273862 (+ 1193us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:34:38.273866 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:34:38.273975 (+ 109us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:34:38.274018 (+ 43us) client_negotiation.cc:769] Sending connection context
0806 18:34:38.274095 (+ 77us) client_negotiation.cc:240] Negotiation successful
0806 18:34:38.274159 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":33}
I20260806 18:34:38.274785 27333 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:34:38.268919 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37854 (local address 127.19.194.61:39189)
0806 18:34:38.269067 (+ 148us) server_negotiation.cc:206] Beginning negotiation
0806 18:34:38.269071 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 18:34:38.269086 (+ 15us) server_negotiation.cc:407] Connection header received
0806 18:34:38.269185 (+ 99us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:34:38.269190 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:34:38.269310 (+ 120us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:34:38.269399 (+ 89us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 18:34:38.269868 (+ 469us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:34:38.270882 (+ 1014us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:34:38.272799 (+ 1917us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:34:38.273554 (+ 755us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:34:38.273652 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:34:38.273796 (+ 144us) server_negotiation.cc:1035] Waiting for connection context
0806 18:34:38.274597 (+ 801us) server_negotiation.cc:299] Negotiation successful
0806 18:34:38.274672 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260806 18:34:38.275338 27217 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 9a38e245bcc34630ba3cc6fb6ca11cb3 [term 1 LEARNER]: Refusing update from remote peer a475b841d65e4ef59cb6d53d3e454091: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20260806 18:34:38.275990 27165 consensus_queue.cc:1035] T 00000000000000000000000000000000 P a475b841d65e4ef59cb6d53d3e454091 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9a38e245bcc34630ba3cc6fb6ca11cb3" member_type: NON_VOTER last_known_addr { host: "127.19.194.61" port: 39189 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260806 18:34:38.277725 27217 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 9a38e245bcc34630ba3cc6fb6ca11cb3 [term 1 LEARNER]: Committing config change with OpId 1.6: config changed from index 5 to 6, NON_VOTER ad824649dcf140339b33a7ec0f8ebdce (127.19.194.60) added. New config: { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "a475b841d65e4ef59cb6d53d3e454091" member_type: VOTER last_known_addr { host: "127.19.194.62" port: 46737 } } peers { permanent_uuid: "9a38e245bcc34630ba3cc6fb6ca11cb3" member_type: NON_VOTER last_known_addr { host: "127.19.194.61" port: 39189 } attrs { promote: true } } peers { permanent_uuid: "ad824649dcf140339b33a7ec0f8ebdce" member_type: NON_VOTER last_known_addr { host: "127.19.194.60" port: 41749 } attrs { promote: true } } }
I20260806 18:34:38.279949 27246 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9a38e245bcc34630ba3cc6fb6ca11cb3 [sys.catalog]: SysCatalogTable state changed. Reason: New leader a475b841d65e4ef59cb6d53d3e454091. Latest consensus state: current_term: 1 leader_uuid: "a475b841d65e4ef59cb6d53d3e454091" committed_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "a475b841d65e4ef59cb6d53d3e454091" member_type: VOTER last_known_addr { host: "127.19.194.62" port: 46737 } } peers { permanent_uuid: "9a38e245bcc34630ba3cc6fb6ca11cb3" member_type: NON_VOTER last_known_addr { host: "127.19.194.61" port: 39189 } attrs { promote: true } } peers { permanent_uuid: "ad824649dcf140339b33a7ec0f8ebdce" member_type: NON_VOTER last_known_addr { host: "127.19.194.60" port: 41749 } attrs { promote: true } } }
I20260806 18:34:38.280059 27246 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9a38e245bcc34630ba3cc6fb6ca11cb3 [sys.catalog]: This master's current role is: LEARNER
I20260806 18:34:38.280146 27246 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9a38e245bcc34630ba3cc6fb6ca11cb3 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "a475b841d65e4ef59cb6d53d3e454091" committed_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "a475b841d65e4ef59cb6d53d3e454091" member_type: VOTER last_known_addr { host: "127.19.194.62" port: 46737 } } peers { permanent_uuid: "9a38e245bcc34630ba3cc6fb6ca11cb3" member_type: NON_VOTER last_known_addr { host: "127.19.194.61" port: 39189 } attrs { promote: true } } peers { permanent_uuid: "ad824649dcf140339b33a7ec0f8ebdce" member_type: NON_VOTER last_known_addr { host: "127.19.194.60" port: 41749 } attrs { promote: true } } }
I20260806 18:34:38.280207 27246 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9a38e245bcc34630ba3cc6fb6ca11cb3 [sys.catalog]: This master's current role is: LEARNER
I20260806 18:34:38.280700 27179 raft_consensus.cc:2229] T 00000000000000000000000000000000 P ad824649dcf140339b33a7ec0f8ebdce [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 18:34:38.280977 27179 raft_consensus.cc:2258] T 00000000000000000000000000000000 P ad824649dcf140339b33a7ec0f8ebdce [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 18:34:38.281081 27179 tablet_replica.cc:330] T 00000000000000000000000000000000 P ad824649dcf140339b33a7ec0f8ebdce: stopping tablet replica
I20260806 18:34:38.281107 27165 raft_consensus.cc:1050] T 00000000000000000000000000000000 P a475b841d65e4ef59cb6d53d3e454091: attempting to promote NON_VOTER 9a38e245bcc34630ba3cc6fb6ca11cb3 to VOTER
I20260806 18:34:38.281497 27165 consensus_queue.cc:237] T 00000000000000000000000000000000 P a475b841d65e4ef59cb6d53d3e454091 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 1.6, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "a475b841d65e4ef59cb6d53d3e454091" member_type: VOTER last_known_addr { host: "127.19.194.62" port: 46737 } } peers { permanent_uuid: "9a38e245bcc34630ba3cc6fb6ca11cb3" member_type: VOTER last_known_addr { host: "127.19.194.61" port: 39189 } attrs { promote: false } } peers { permanent_uuid: "ad824649dcf140339b33a7ec0f8ebdce" member_type: NON_VOTER last_known_addr { host: "127.19.194.60" port: 41749 } attrs { promote: true } }
I20260806 18:34:38.283440 27217 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 9a38e245bcc34630ba3cc6fb6ca11cb3 [term 1 LEARNER]: Refusing update from remote peer a475b841d65e4ef59cb6d53d3e454091: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20260806 18:34:38.284478 27166 consensus_queue.cc:1035] T 00000000000000000000000000000000 P a475b841d65e4ef59cb6d53d3e454091 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9a38e245bcc34630ba3cc6fb6ca11cb3" member_type: VOTER last_known_addr { host: "127.19.194.61" port: 39189 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
W20260806 18:34:38.285269 26579 consensus_peers.cc:487] T 00000000000000000000000000000000 P a475b841d65e4ef59cb6d53d3e454091 -> Peer ad824649dcf140339b33a7ec0f8ebdce (127.19.194.60:41749): Couldn't send request to peer ad824649dcf140339b33a7ec0f8ebdce. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on Master. This is attempt 1: this message will repeat every 5th retry.
I20260806 18:34:38.288044 27217 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 9a38e245bcc34630ba3cc6fb6ca11cb3 [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, 9a38e245bcc34630ba3cc6fb6ca11cb3 (127.19.194.61) changed from NON_VOTER to VOTER. New config: { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "a475b841d65e4ef59cb6d53d3e454091" member_type: VOTER last_known_addr { host: "127.19.194.62" port: 46737 } } peers { permanent_uuid: "9a38e245bcc34630ba3cc6fb6ca11cb3" member_type: VOTER last_known_addr { host: "127.19.194.61" port: 39189 } attrs { promote: false } } peers { permanent_uuid: "ad824649dcf140339b33a7ec0f8ebdce" member_type: NON_VOTER last_known_addr { host: "127.19.194.60" port: 41749 } attrs { promote: true } } }
I20260806 18:34:38.287750 27358 raft_consensus.cc:2932] T 00000000000000000000000000000000 P a475b841d65e4ef59cb6d53d3e454091 [term 1 LEADER]: Committing config change with OpId 1.7: config changed from index 6 to 7, 9a38e245bcc34630ba3cc6fb6ca11cb3 (127.19.194.61) changed from NON_VOTER to VOTER. New config: { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "a475b841d65e4ef59cb6d53d3e454091" member_type: VOTER last_known_addr { host: "127.19.194.62" port: 46737 } } peers { permanent_uuid: "9a38e245bcc34630ba3cc6fb6ca11cb3" member_type: VOTER last_known_addr { host: "127.19.194.61" port: 39189 } attrs { promote: false } } peers { permanent_uuid: "ad824649dcf140339b33a7ec0f8ebdce" member_type: NON_VOTER last_known_addr { host: "127.19.194.60" port: 41749 } attrs { promote: true } } }
I20260806 18:34:38.290982 27357 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9a38e245bcc34630ba3cc6fb6ca11cb3 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "a475b841d65e4ef59cb6d53d3e454091" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "a475b841d65e4ef59cb6d53d3e454091" member_type: VOTER last_known_addr { host: "127.19.194.62" port: 46737 } } peers { permanent_uuid: "9a38e245bcc34630ba3cc6fb6ca11cb3" member_type: VOTER last_known_addr { host: "127.19.194.61" port: 39189 } attrs { promote: false } } peers { permanent_uuid: "ad824649dcf140339b33a7ec0f8ebdce" member_type: NON_VOTER last_known_addr { host: "127.19.194.60" port: 41749 } attrs { promote: true } } }
I20260806 18:34:38.291106 27357 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9a38e245bcc34630ba3cc6fb6ca11cb3 [sys.catalog]: This master's current role is: FOLLOWER
I20260806 18:34:38.291519 27358 sys_catalog.cc:455] T 00000000000000000000000000000000 P a475b841d65e4ef59cb6d53d3e454091 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "a475b841d65e4ef59cb6d53d3e454091" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "a475b841d65e4ef59cb6d53d3e454091" member_type: VOTER last_known_addr { host: "127.19.194.62" port: 46737 } } peers { permanent_uuid: "9a38e245bcc34630ba3cc6fb6ca11cb3" member_type: VOTER last_known_addr { host: "127.19.194.61" port: 39189 } attrs { promote: false } } peers { permanent_uuid: "ad824649dcf140339b33a7ec0f8ebdce" member_type: NON_VOTER last_known_addr { host: "127.19.194.60" port: 41749 } attrs { promote: true } } }
I20260806 18:34:38.291496 27165 sys_catalog.cc:455] T 00000000000000000000000000000000 P a475b841d65e4ef59cb6d53d3e454091 [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "a475b841d65e4ef59cb6d53d3e454091" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "a475b841d65e4ef59cb6d53d3e454091" member_type: VOTER last_known_addr { host: "127.19.194.62" port: 46737 } } peers { permanent_uuid: "9a38e245bcc34630ba3cc6fb6ca11cb3" member_type: VOTER last_known_addr { host: "127.19.194.61" port: 39189 } attrs { promote: false } } peers { permanent_uuid: "ad824649dcf140339b33a7ec0f8ebdce" member_type: NON_VOTER last_known_addr { host: "127.19.194.60" port: 41749 } attrs { promote: true } } }
I20260806 18:34:38.291632 27358 sys_catalog.cc:458] T 00000000000000000000000000000000 P a475b841d65e4ef59cb6d53d3e454091 [sys.catalog]: This master's current role is: LEADER
I20260806 18:34:38.291647 27165 sys_catalog.cc:458] T 00000000000000000000000000000000 P a475b841d65e4ef59cb6d53d3e454091 [sys.catalog]: This master's current role is: LEADER
I20260806 18:34:38.297617 27179 master.cc:552] Master@127.19.194.60:41749 shutdown complete.
I20260806 18:34:38.297672 27179 master_runner.cc:305] Clearing existing system tablet
I20260806 18:34:38.297914 27179 fs_manager.cc:359] Metadata directory not provided
I20260806 18:34:38.297966 27179 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 18:34:38.299271 27179 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.002s sys 0.000s
I20260806 18:34:38.300273 27361 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 18:34:38.300542 27179 fs_report.cc:383] FS layout report
--------------------
wal directory:
metadata directory:
1 data directories: /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786041246094603-20232-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 18:34:38.300673 27179 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260806 18:34:38.300705 27179 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786041246094603-20232-0/minicluster-data/master-2/data,/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786041246094603-20232-0/minicluster-data/master-2/wal
uuid: "ad824649dcf140339b33a7ec0f8ebdce"
format_stamp: "Formatted at 2026-08-06 18:34:38 on dist-test-slave-1xrh"
server_key: "fb2265fe9667197d32b9a03bb06ca4ae"
server_key_iv: "cf2f54a4fac4e1f3dcca4fb5c530f693"
server_key_version: "encryptionkey@0"
I20260806 18:34:38.302661 27179 ts_tablet_manager.cc:1905] T 00000000000000000000000000000000 P ad824649dcf140339b33a7ec0f8ebdce: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 18:34:38.305235 27179 ts_tablet_manager.cc:1918] T 00000000000000000000000000000000 P ad824649dcf140339b33a7ec0f8ebdce: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId unknown
I20260806 18:34:38.305392 27179 log.cc:1196] T 00000000000000000000000000000000 P ad824649dcf140339b33a7ec0f8ebdce: Deleting WAL directory at /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786041246094603-20232-0/minicluster-data/master-2/wal/wals/00000000000000000000000000000000
I20260806 18:34:38.305656 27179 ts_tablet_manager.cc:1939] T 00000000000000000000000000000000 P ad824649dcf140339b33a7ec0f8ebdce: Deleting consensus metadata
I20260806 18:34:38.305820 27179 master_runner.cc:315] Copying system tablet from 127.19.194.62:46737
I20260806 18:34:38.306943 27179 tablet_copy_client.cc:316] T 00000000000000000000000000000000 P ad824649dcf140339b33a7ec0f8ebdce: tablet copy: Beginning tablet copy session from remote peer at address 127.19.194.62:46737
I20260806 18:34:38.317302 26636 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:34:38.307578 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44442 (local address 127.19.194.62:46737)
0806 18:34:38.307695 (+ 117us) server_negotiation.cc:206] Beginning negotiation
0806 18:34:38.307699 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 18:34:38.307900 (+ 201us) server_negotiation.cc:407] Connection header received
0806 18:34:38.308010 (+ 110us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:34:38.308013 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:34:38.308064 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:34:38.308155 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:34:38.308688 (+ 533us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:34:38.309472 (+ 784us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:34:38.310726 (+ 1254us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:34:38.311065 (+ 339us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:34:38.311225 (+ 160us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:34:38.313265 (+ 2040us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:34:38.313270 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:34:38.313272 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 18:34:38.313301 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:34:38.315361 (+ 2060us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:34:38.315890 (+ 529us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:34:38.315893 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:34:38.315895 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:34:38.315993 (+ 98us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:34:38.316312 (+ 319us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:34:38.316318 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:34:38.316321 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:34:38.316586 (+ 265us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:34:38.316698 (+ 112us) server_negotiation.cc:1035] Waiting for connection context
0806 18:34:38.317012 (+ 314us) server_negotiation.cc:299] Negotiation successful
0806 18:34:38.317141 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260806 18:34:38.317263 27367 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:34:38.307498 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.194.62:46737 (local address 127.0.0.1:44442)
0806 18:34:38.307800 (+ 302us) negotiation.cc:107] Waiting for socket to connect
0806 18:34:38.307814 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0806 18:34:38.307929 (+ 115us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:34:38.308205 (+ 276us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:34:38.308208 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:34:38.308419 (+ 211us) client_negotiation.cc:189] Negotiated authn=SASL
0806 18:34:38.308560 (+ 141us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:34:38.308568 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:34:38.309618 (+ 1050us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:34:38.309622 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:34:38.310583 (+ 961us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:34:38.310595 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:34:38.311230 (+ 635us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:34:38.311234 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:34:38.311336 (+ 102us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:34:38.312472 (+ 1136us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 18:34:38.312487 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0806 18:34:38.313061 (+ 574us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 18:34:38.315488 (+ 2427us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:34:38.315492 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:34:38.315494 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:34:38.315769 (+ 275us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:34:38.316099 (+ 330us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:34:38.316103 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:34:38.316104 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:34:38.316202 (+ 98us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:34:38.316703 (+ 501us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 18:34:38.316706 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 18:34:38.316811 (+ 105us) client_negotiation.cc:769] Sending connection context
0806 18:34:38.316928 (+ 117us) client_negotiation.cc:240] Negotiation successful
0806 18:34:38.317075 (+ 147us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":235,"thread_start_us":94,"threads_started":1}
I20260806 18:34:38.318205 26617 tablet_copy_service.cc:140] P a475b841d65e4ef59cb6d53d3e454091: Received BeginTabletCopySession request for tablet 00000000000000000000000000000000 from peer ad824649dcf140339b33a7ec0f8ebdce ({username='oryx', principal='oryx/127.19.194.60@KRBTEST.COM'} at 127.0.0.1:44442)
I20260806 18:34:38.318306 26617 tablet_copy_service.cc:161] P a475b841d65e4ef59cb6d53d3e454091: Beginning new tablet copy session on tablet 00000000000000000000000000000000 from peer ad824649dcf140339b33a7ec0f8ebdce at {username='oryx', principal='oryx/127.19.194.60@KRBTEST.COM'} at 127.0.0.1:44442: session id = ad824649dcf140339b33a7ec0f8ebdce-00000000000000000000000000000000
I20260806 18:34:38.319491 26617 tablet_copy_source_session.cc:205] T 00000000000000000000000000000000 P a475b841d65e4ef59cb6d53d3e454091: Tablet Copy: opened 0 blocks and 1 log segments
I20260806 18:34:38.320755 27179 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total
I20260806 18:34:38.323426 27179 tablet_copy_client.cc:791] T 00000000000000000000000000000000 P ad824649dcf140339b33a7ec0f8ebdce: tablet copy: Starting download of 0 data blocks...
I20260806 18:34:38.323618 27179 tablet_copy_client.cc:655] T 00000000000000000000000000000000 P ad824649dcf140339b33a7ec0f8ebdce: tablet copy: Starting download of 1 WAL segments...
I20260806 18:34:38.325212 27179 tablet_copy_client.cc:531] T 00000000000000000000000000000000 P ad824649dcf140339b33a7ec0f8ebdce: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260806 18:34:38.327015 26617 tablet_copy_service.cc:342] P a475b841d65e4ef59cb6d53d3e454091: Request end of tablet copy session ad824649dcf140339b33a7ec0f8ebdce-00000000000000000000000000000000 received from {username='oryx', principal='oryx/127.19.194.60@KRBTEST.COM'} at 127.0.0.1:44442
I20260806 18:34:38.327102 26617 tablet_copy_service.cc:433] P a475b841d65e4ef59cb6d53d3e454091: ending tablet copy session ad824649dcf140339b33a7ec0f8ebdce-00000000000000000000000000000000 on tablet 00000000000000000000000000000000 with peer ad824649dcf140339b33a7ec0f8ebdce
I20260806 18:34:38.335251 27179 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 18:34:38.336937 27368 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:34:38.337131 27369 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:34:38.337406 27371 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 18:34:38.337620 27179 server_base.cc:1034] running on GCE node
I20260806 18:34:38.337833 27179 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 18:34:38.338120 27179 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 18:34:38.339250 27179 hybrid_clock.cc:648] HybridClock initialized: now 1786041278339238 us; error 36 us; skew 500 ppm
Aug 06 18:34:38 dist-test-slave-1xrh krb5kdc[26549](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041278, etypes {rep=17 tkt=17 ses=17}, oryx/127.19.194.60@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 18:34:38.340907 27179 init.cc:377] Logged in from keytab as oryx/127.19.194.60@KRBTEST.COM (short username oryx)
I20260806 18:34:38.341398 27179 webserver.cc:460] Webserver started at http://127.19.194.60:35653/ using document root <none> and password file <none>
I20260806 18:34:38.341614 27179 fs_manager.cc:359] Metadata directory not provided
I20260806 18:34:38.341656 27179 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 18:34:38.343050 27179 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260806 18:34:38.344332 27378 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 18:34:38.344615 27179 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260806 18:34:38.344691 27179 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786041246094603-20232-0/minicluster-data/master-2/data,/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786041246094603-20232-0/minicluster-data/master-2/wal
uuid: "ad824649dcf140339b33a7ec0f8ebdce"
format_stamp: "Formatted at 2026-08-06 18:34:38 on dist-test-slave-1xrh"
server_key: "fb2265fe9667197d32b9a03bb06ca4ae"
server_key_iv: "cf2f54a4fac4e1f3dcca4fb5c530f693"
server_key_version: "encryptionkey@0"
I20260806 18:34:38.344766 27179 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786041246094603-20232-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786041246094603-20232-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786041246094603-20232-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 18:34:38.358143 27179 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 18:34:38.360630 27179 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 18:34:38.366355 27179 rpc_server.cc:307] RPC server started. Bound to: 127.19.194.60:41749
I20260806 18:34:38.366391 27431 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.194.60:41749 every 8 connection(s)
I20260806 18:34:38.366981 27179 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786041246094603-20232-0/minicluster-data/master-2/data/info.pb
I20260806 18:34:38.368817 27432 sys_catalog.cc:263] Verifying existing consensus state
I20260806 18:34:38.369961 27432 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ad824649dcf140339b33a7ec0f8ebdce: Bootstrap starting.
W20260806 18:34:38.380295 27355 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 18:34:38.380738 27355 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 18:34:38.380839 27355 flags.cc:425] Enabled unsafe flag: --never_fsync=true
I20260806 18:34:38.382115 27432 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ad824649dcf140339b33a7ec0f8ebdce: Bootstrap replayed 1/1 log segments. Stats: ops{read=7 overwritten=0 applied=7 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260806 18:34:38.382483 27432 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ad824649dcf140339b33a7ec0f8ebdce: Bootstrap complete.
I20260806 18:34:38.383056 27432 raft_consensus.cc:348] T 00000000000000000000000000000000 P ad824649dcf140339b33a7ec0f8ebdce [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "a475b841d65e4ef59cb6d53d3e454091" member_type: VOTER last_known_addr { host: "127.19.194.62" port: 46737 } } peers { permanent_uuid: "9a38e245bcc34630ba3cc6fb6ca11cb3" member_type: VOTER last_known_addr { host: "127.19.194.61" port: 39189 } attrs { promote: false } } peers { permanent_uuid: "ad824649dcf140339b33a7ec0f8ebdce" member_type: NON_VOTER last_known_addr { host: "127.19.194.60" port: 41749 } attrs { promote: true } }
I20260806 18:34:38.383244 27432 raft_consensus.cc:729] T 00000000000000000000000000000000 P ad824649dcf140339b33a7ec0f8ebdce [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: ad824649dcf140339b33a7ec0f8ebdce, State: Initialized, Role: LEARNER
I20260806 18:34:38.383358 27432 consensus_queue.cc:260] T 00000000000000000000000000000000 P ad824649dcf140339b33a7ec0f8ebdce [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 7, Last appended: 1.7, Last appended by leader: 7, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "a475b841d65e4ef59cb6d53d3e454091" member_type: VOTER last_known_addr { host: "127.19.194.62" port: 46737 } } peers { permanent_uuid: "9a38e245bcc34630ba3cc6fb6ca11cb3" member_type: VOTER last_known_addr { host: "127.19.194.61" port: 39189 } attrs { promote: false } } peers { permanent_uuid: "ad824649dcf140339b33a7ec0f8ebdce" member_type: NON_VOTER last_known_addr { host: "127.19.194.60" port: 41749 } attrs { promote: true } }
I20260806 18:34:38.383823 27432 sys_catalog.cc:564] T 00000000000000000000000000000000 P ad824649dcf140339b33a7ec0f8ebdce [sys.catalog]: configured and running, proceeding with master startup.
I20260806 18:34:38.383821 27438 sys_catalog.cc:455] T 00000000000000000000000000000000 P ad824649dcf140339b33a7ec0f8ebdce [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "a475b841d65e4ef59cb6d53d3e454091" member_type: VOTER last_known_addr { host: "127.19.194.62" port: 46737 } } peers { permanent_uuid: "9a38e245bcc34630ba3cc6fb6ca11cb3" member_type: VOTER last_known_addr { host: "127.19.194.61" port: 39189 } attrs { promote: false } } peers { permanent_uuid: "ad824649dcf140339b33a7ec0f8ebdce" member_type: NON_VOTER last_known_addr { host: "127.19.194.60" port: 41749 } attrs { promote: true } } }
I20260806 18:34:38.384075 27438 sys_catalog.cc:458] T 00000000000000000000000000000000 P ad824649dcf140339b33a7ec0f8ebdce [sys.catalog]: This master's current role is: LEARNER
W20260806 18:34:38.386317 27355 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 18:34:38.386390 27355 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 18:34:38.386497 27355 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.19.194.1
I20260806 18:34:38.387054 27449 catalog_manager.cc:1259] Loaded cluster ID: 3fbbab1f31df424cb2df0f828bded742
I20260806 18:34:38.387177 27449 catalog_manager.cc:1552] T 00000000000000000000000000000000 P ad824649dcf140339b33a7ec0f8ebdce: loading cluster ID for follower catalog manager: success
I20260806 18:34:38.389322 27449 catalog_manager.cc:1574] T 00000000000000000000000000000000 P ad824649dcf140339b33a7ec0f8ebdce: acquiring CA information for follower catalog manager: success
I20260806 18:34:38.390641 27449 catalog_manager.cc:1602] T 00000000000000000000000000000000 P ad824649dcf140339b33a7ec0f8ebdce: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260806 18:34:38.392932 27355 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.194.20:44897
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786041246094603-20232-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786041246094603-20232-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786041246094603-20232-0/krb5kdc/oryx_127.19.194.1.keytab
--principal=oryx/127.19.194.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.19.194.1:33871
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786041246094603-20232-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.19.194.1
--webserver_port=33093
--tserver_master_addrs=127.19.194.62:46737,127.19.194.61:39189,127.19.194.60:41749
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27355
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.19.194.1
--log_dir=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786041246094603-20232-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 17:59:13 UTC on ccycloud.quasar-wvsnzx.root.comops.site
build id 4571028
I20260806 18:34:38.393915 27355 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 18:34:38.394919 27355 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260806 18:34:38.405305 27355 server_base.cc:1034] running on GCE node
W20260806 18:34:38.405215 27454 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:34:38.405239 27451 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:34:38.405221 27452 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 18:34:38.405990 27355 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 18:34:38.406553 27355 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 18:34:38.407814 27355 hybrid_clock.cc:648] HybridClock initialized: now 1786041278407792 us; error 29 us; skew 500 ppm
Aug 06 18:34:38 dist-test-slave-1xrh krb5kdc[26549](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041278, etypes {rep=17 tkt=17 ses=17}, oryx/127.19.194.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 18:34:38.410665 27355 init.cc:377] Logged in from keytab as oryx/127.19.194.1@KRBTEST.COM (short username oryx)
I20260806 18:34:38.411633 27355 webserver.cc:460] Webserver started at http://127.19.194.1:33093/ using document root <none> and password file <none>
I20260806 18:34:38.412237 27355 fs_manager.cc:359] Metadata directory not provided
I20260806 18:34:38.412364 27355 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 18:34:38.416210 27355 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.004s sys 0.000s
I20260806 18:34:38.418171 27462 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 18:34:38.419427 27355 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260806 18:34:38.419565 27355 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786041246094603-20232-0/minicluster-data/ts-0/data,/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786041246094603-20232-0/minicluster-data/ts-0/wal
uuid: "a7dbdd68b24540cda5ada7e1e0d73ccd"
format_stamp: "Formatted at 2026-08-06 18:34:37 on dist-test-slave-1xrh"
server_key: "d129966921ed58c57baacbf98cc8e63e"
server_key_iv: "62b325fd36ca0563239daa6ce37a24f1"
server_key_version: "encryptionkey@0"
I20260806 18:34:38.420107 27355 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786041246094603-20232-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786041246094603-20232-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786041246094603-20232-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 18:34:38.421058 26928 heartbeater.cc:503] Master 127.19.194.62:46737 was elected leader, sending a full tablet report...
I20260806 18:34:38.443296 27355 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 18:34:38.446748 27355 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 18:34:38.446943 27355 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 18:34:38.447605 27355 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 18:34:38.448475 27355 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 18:34:38.448558 27355 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 18:34:38.448627 27355 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 18:34:38.448660 27355 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 18:34:38.457902 27355 rpc_server.cc:307] RPC server started. Bound to: 127.19.194.1:33871
I20260806 18:34:38.457953 27576 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.194.1:33871 every 8 connection(s)
I20260806 18:34:38.458961 27355 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786041246094603-20232-0/minicluster-data/ts-0/data/info.pb
I20260806 18:34:38.465870 20232 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJW69tn/build/debug/bin/kudu as pid 27355
I20260806 18:34:38.466073 20232 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJW69tn/build/debug/bin/kudu with pid 26795
Aug 06 18:34:38 dist-test-slave-1xrh krb5kdc[26549](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041278, etypes {rep=17 tkt=17 ses=17}, oryx/127.19.194.1@KRBTEST.COM for oryx/127.19.194.61@KRBTEST.COM
Aug 06 18:34:38 dist-test-slave-1xrh krb5kdc[26549](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041278, etypes {rep=17 tkt=17 ses=17}, oryx/127.19.194.1@KRBTEST.COM for oryx/127.19.194.60@KRBTEST.COM
Aug 06 18:34:38 dist-test-slave-1xrh krb5kdc[26549](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041278, etypes {rep=17 tkt=17 ses=17}, oryx/127.19.194.1@KRBTEST.COM for oryx/127.19.194.62@KRBTEST.COM
I20260806 18:34:38.476547 27333 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:34:38.460802 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.194.1:47445 (local address 127.19.194.61:39189)
0806 18:34:38.461059 (+ 257us) server_negotiation.cc:206] Beginning negotiation
0806 18:34:38.461065 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 18:34:38.461842 (+ 777us) server_negotiation.cc:407] Connection header received
0806 18:34:38.463123 (+ 1281us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:34:38.463127 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:34:38.463195 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:34:38.463252 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:34:38.464839 (+ 1587us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:34:38.465640 (+ 801us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:34:38.468143 (+ 2503us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:34:38.468508 (+ 365us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:34:38.468595 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:34:38.472311 (+ 3716us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:34:38.472317 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:34:38.472324 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 18:34:38.472359 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:34:38.474078 (+ 1719us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:34:38.474719 (+ 641us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:34:38.474723 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:34:38.474724 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:34:38.474822 (+ 98us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:34:38.475317 (+ 495us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:34:38.475323 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:34:38.475328 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:34:38.475622 (+ 294us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:34:38.475703 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0806 18:34:38.476253 (+ 550us) server_negotiation.cc:299] Negotiation successful
0806 18:34:38.476371 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":125}
I20260806 18:34:38.477517 27583 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:34:38.461274 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.194.61:39189 (local address 127.19.194.1:47445)
0806 18:34:38.461704 (+ 430us) negotiation.cc:107] Waiting for socket to connect
0806 18:34:38.461747 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0806 18:34:38.462881 (+ 1134us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:34:38.463424 (+ 543us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:34:38.463427 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:34:38.464100 (+ 673us) client_negotiation.cc:189] Negotiated authn=SASL
0806 18:34:38.464675 (+ 575us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:34:38.464688 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:34:38.465813 (+ 1125us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:34:38.465816 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:34:38.468004 (+ 2188us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:34:38.468018 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:34:38.468844 (+ 826us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:34:38.468854 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:34:38.469143 (+ 289us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:34:38.469953 (+ 810us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 18:34:38.469984 (+ 31us) client_negotiation.cc:656] calling sasl_client_start()
0806 18:34:38.472126 (+ 2142us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 18:34:38.474213 (+ 2087us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:34:38.474219 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:34:38.474234 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:34:38.474585 (+ 351us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:34:38.474929 (+ 344us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:34:38.474932 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:34:38.474935 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:34:38.475177 (+ 242us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:34:38.475787 (+ 610us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 18:34:38.475792 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 18:34:38.476010 (+ 218us) client_negotiation.cc:769] Sending connection context
0806 18:34:38.476192 (+ 182us) client_negotiation.cc:240] Negotiation successful
0806 18:34:38.476526 (+ 334us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":365,"mutex_wait_us":134,"spinlock_wait_cycles":28032,"thread_start_us":108,"threads_started":1}
I20260806 18:34:38.478699 20232 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJW69tn/build/debug/bin/kudu
/tmp/dist-test-taskJW69tn/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786041246094603-20232-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786041246094603-20232-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786041246094603-20232-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786041246094603-20232-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.19.194.2:44715
--local_ip_for_outbound_sockets=127.19.194.2
--tserver_master_addrs=127.19.194.62:46737
--webserver_port=35523
--webserver_interface=127.19.194.2
--keytab_file=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786041246094603-20232-0/krb5kdc/oryx_127.19.194.2.keytab
--principal=oryx/127.19.194.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.19.194.20:44897
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.19.194.62:46737,127.19.194.61:39189
--tserver_master_addrs=127.19.194.62:46737,127.19.194.61:39189,127.19.194.60:41749 with env {KRB5CCNAME=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786041246094603-20232-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786041246094603-20232-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786041246094603-20232-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260806 18:34:38.478988 27580 heartbeater.cc:347] Connected to a master server at 127.19.194.61:39189
I20260806 18:34:38.479257 27580 heartbeater.cc:464] Registering TS with master...
I20260806 18:34:38.479859 27580 heartbeater.cc:511] Master 127.19.194.61:39189 requested a full tablet report, sending...
I20260806 18:34:38.481040 26636 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:34:38.460818 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.194.1:58425 (local address 127.19.194.62:46737)
0806 18:34:38.460988 (+ 170us) server_negotiation.cc:206] Beginning negotiation
0806 18:34:38.460993 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 18:34:38.461842 (+ 849us) server_negotiation.cc:407] Connection header received
0806 18:34:38.463090 (+ 1248us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:34:38.463095 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:34:38.463185 (+ 90us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:34:38.463269 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:34:38.464839 (+ 1570us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:34:38.465641 (+ 802us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:34:38.473750 (+ 8109us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:34:38.474128 (+ 378us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:34:38.474208 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:34:38.476948 (+ 2740us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:34:38.476955 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:34:38.476973 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 18:34:38.477007 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:34:38.478824 (+ 1817us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:34:38.479607 (+ 783us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:34:38.479611 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:34:38.479613 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:34:38.479705 (+ 92us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:34:38.480097 (+ 392us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:34:38.480100 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:34:38.480102 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:34:38.480318 (+ 216us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:34:38.480406 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0806 18:34:38.480749 (+ 343us) server_negotiation.cc:299] Negotiation successful
0806 18:34:38.480898 (+ 149us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260806 18:34:38.481505 27199 ts_manager.cc:194] Registered new tserver with Master: a7dbdd68b24540cda5ada7e1e0d73ccd (127.19.194.1:33871)
I20260806 18:34:38.481812 27582 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:34:38.461236 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.194.62:46737 (local address 127.19.194.1:58425)
0806 18:34:38.461695 (+ 459us) negotiation.cc:107] Waiting for socket to connect
0806 18:34:38.461746 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0806 18:34:38.462860 (+ 1114us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:34:38.463416 (+ 556us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:34:38.463422 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:34:38.464112 (+ 690us) client_negotiation.cc:189] Negotiated authn=SASL
0806 18:34:38.464675 (+ 563us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:34:38.464688 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:34:38.465813 (+ 1125us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:34:38.465816 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:34:38.473593 (+ 7777us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:34:38.473607 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:34:38.474382 (+ 775us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:34:38.474384 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:34:38.474488 (+ 104us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:34:38.474922 (+ 434us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 18:34:38.474936 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0806 18:34:38.476768 (+ 1832us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 18:34:38.479190 (+ 2422us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:34:38.479193 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:34:38.479195 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:34:38.479484 (+ 289us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:34:38.479836 (+ 352us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:34:38.479839 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:34:38.479841 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:34:38.479983 (+ 142us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:34:38.480468 (+ 485us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 18:34:38.480471 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 18:34:38.480558 (+ 87us) client_negotiation.cc:769] Sending connection context
0806 18:34:38.481416 (+ 858us) client_negotiation.cc:240] Negotiation successful
0806 18:34:38.481558 (+ 142us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":267,"mutex_wait_us":6730,"spinlock_wait_cycles":98304,"thread_start_us":192,"threads_started":1}
I20260806 18:34:38.483031 27579 heartbeater.cc:347] Connected to a master server at 127.19.194.62:46737
I20260806 18:34:38.483099 27584 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:34:38.462358 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.194.60:41749 (local address 127.19.194.1:37441)
0806 18:34:38.462609 (+ 251us) negotiation.cc:107] Waiting for socket to connect
0806 18:34:38.462622 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0806 18:34:38.462860 (+ 238us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:34:38.463605 (+ 745us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:34:38.463607 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:34:38.464103 (+ 496us) client_negotiation.cc:189] Negotiated authn=SASL
0806 18:34:38.464673 (+ 570us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:34:38.464686 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:34:38.470864 (+ 6178us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:34:38.470871 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:34:38.471896 (+ 1025us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:34:38.471908 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:34:38.473073 (+ 1165us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:34:38.473080 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:34:38.473482 (+ 402us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:34:38.474019 (+ 537us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 18:34:38.474039 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0806 18:34:38.478096 (+ 4057us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 18:34:38.481161 (+ 3065us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:34:38.481166 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:34:38.481169 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:34:38.481425 (+ 256us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:34:38.481776 (+ 351us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:34:38.481779 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:34:38.481781 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:34:38.481879 (+ 98us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:34:38.482416 (+ 537us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 18:34:38.482422 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 18:34:38.482531 (+ 109us) client_negotiation.cc:769] Sending connection context
0806 18:34:38.482637 (+ 106us) client_negotiation.cc:240] Negotiation successful
0806 18:34:38.482797 (+ 160us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":186,"mutex_wait_us":52,"thread_start_us":91,"threads_started":1}
I20260806 18:34:38.483170 27579 heartbeater.cc:464] Registering TS with master...
I20260806 18:34:38.483361 27579 heartbeater.cc:511] Master 127.19.194.62:46737 requested a full tablet report, sending...
I20260806 18:34:38.484118 27585 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:34:38.462464 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.194.1:37441 (local address 127.19.194.60:41749)
0806 18:34:38.462734 (+ 270us) server_negotiation.cc:206] Beginning negotiation
0806 18:34:38.462737 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 18:34:38.462751 (+ 14us) server_negotiation.cc:407] Connection header received
0806 18:34:38.463398 (+ 647us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:34:38.463402 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:34:38.463452 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:34:38.463565 (+ 113us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:34:38.464831 (+ 1266us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:34:38.465526 (+ 695us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:34:38.472040 (+ 6514us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:34:38.472508 (+ 468us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:34:38.472908 (+ 400us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:34:38.479004 (+ 6096us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:34:38.479010 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:34:38.479016 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 18:34:38.479051 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:34:38.481021 (+ 1970us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:34:38.481559 (+ 538us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:34:38.481564 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:34:38.481566 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:34:38.481661 (+ 95us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:34:38.482004 (+ 343us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:34:38.482010 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:34:38.482014 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:34:38.482267 (+ 253us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:34:38.482357 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0806 18:34:38.483541 (+ 1184us) server_negotiation.cc:299] Negotiation successful
0806 18:34:38.483757 (+ 216us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":205,"thread_start_us":86,"threads_started":1}
I20260806 18:34:38.484265 26597 ts_manager.cc:194] Re-registered known tserver with Master: a7dbdd68b24540cda5ada7e1e0d73ccd (127.19.194.1:33871)
I20260806 18:34:38.484889 27577 heartbeater.cc:347] Connected to a master server at 127.19.194.60:41749
I20260806 18:34:38.484994 27577 heartbeater.cc:464] Registering TS with master...
I20260806 18:34:38.485074 26597 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.19.194.1@KRBTEST.COM'} at 127.19.194.1:58425
I20260806 18:34:38.485271 27577 heartbeater.cc:511] Master 127.19.194.60:41749 requested a full tablet report, sending...
I20260806 18:34:38.486521 27395 ts_manager.cc:194] Registered new tserver with Master: a7dbdd68b24540cda5ada7e1e0d73ccd (127.19.194.1:33871)
W20260806 18:34:38.591882 27586 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 18:34:38.592103 27586 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 18:34:38.592141 27586 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 18:34:38.596055 27586 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 18:34:38.596104 27586 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 18:34:38.596195 27586 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.19.194.2
I20260806 18:34:38.601485 27586 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.194.20:44897
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786041246094603-20232-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786041246094603-20232-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786041246094603-20232-0/krb5kdc/oryx_127.19.194.2.keytab
--principal=oryx/127.19.194.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.19.194.2:44715
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786041246094603-20232-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.19.194.2
--webserver_port=35523
--tserver_master_addrs=127.19.194.62:46737,127.19.194.61:39189,127.19.194.60:41749
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27586
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.19.194.2
--log_dir=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786041246094603-20232-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 17:59:13 UTC on ccycloud.quasar-wvsnzx.root.comops.site
build id 4571028
I20260806 18:34:38.602468 27586 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 18:34:38.603456 27586 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 18:34:38.613188 27592 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:34:38.613355 27595 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:34:38.613278 27593 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 18:34:38.613622 27586 server_base.cc:1034] running on GCE node
I20260806 18:34:38.614055 27586 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 18:34:38.614745 27586 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 18:34:38.615938 27586 hybrid_clock.cc:648] HybridClock initialized: now 1786041278615898 us; error 50 us; skew 500 ppm
Aug 06 18:34:38 dist-test-slave-1xrh krb5kdc[26549](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041278, etypes {rep=17 tkt=17 ses=17}, oryx/127.19.194.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 18:34:38.618677 27586 init.cc:377] Logged in from keytab as oryx/127.19.194.2@KRBTEST.COM (short username oryx)
I20260806 18:34:38.619560 27586 webserver.cc:460] Webserver started at http://127.19.194.2:35523/ using document root <none> and password file <none>
I20260806 18:34:38.620039 27586 fs_manager.cc:359] Metadata directory not provided
I20260806 18:34:38.620108 27586 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 18:34:38.624013 27586 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.003s sys 0.000s
I20260806 18:34:38.626096 27602 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 18:34:38.627347 27586 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260806 18:34:38.627480 27586 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786041246094603-20232-0/minicluster-data/ts-1/data,/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786041246094603-20232-0/minicluster-data/ts-1/wal
uuid: "a5630d59c6324f3b87d9d2a1f4024c45"
format_stamp: "Formatted at 2026-08-06 18:34:37 on dist-test-slave-1xrh"
server_key: "2c031e1e1419e1611cd5cef94f10e964"
server_key_iv: "e938fb4774cac2c1c426a6f635e5104b"
server_key_version: "encryptionkey@0"
I20260806 18:34:38.628021 27586 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786041246094603-20232-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786041246094603-20232-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786041246094603-20232-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 18:34:38.631944 27065 heartbeater.cc:503] Master 127.19.194.62:46737 was elected leader, sending a full tablet report...
I20260806 18:34:38.643009 27586 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 18:34:38.646059 27586 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 18:34:38.646261 27586 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 18:34:38.646843 27586 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 18:34:38.647761 27586 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 18:34:38.647840 27586 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 18:34:38.647907 27586 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 18:34:38.647957 27586 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 18:34:38.656610 27586 rpc_server.cc:307] RPC server started. Bound to: 127.19.194.2:44715
I20260806 18:34:38.656642 27716 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.194.2:44715 every 8 connection(s)
I20260806 18:34:38.657727 27586 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786041246094603-20232-0/minicluster-data/ts-1/data/info.pb
I20260806 18:34:38.666209 20232 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJW69tn/build/debug/bin/kudu as pid 27586
I20260806 18:34:38.666388 20232 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJW69tn/build/debug/bin/kudu with pid 26933
Aug 06 18:34:38 dist-test-slave-1xrh krb5kdc[26549](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041278, etypes {rep=17 tkt=17 ses=17}, oryx/127.19.194.2@KRBTEST.COM for oryx/127.19.194.62@KRBTEST.COM
Aug 06 18:34:38 dist-test-slave-1xrh krb5kdc[26549](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041278, etypes {rep=17 tkt=17 ses=17}, oryx/127.19.194.2@KRBTEST.COM for oryx/127.19.194.60@KRBTEST.COM
I20260806 18:34:38.673695 26636 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:34:38.660211 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.194.2:39805 (local address 127.19.194.62:46737)
0806 18:34:38.660367 (+ 156us) server_negotiation.cc:206] Beginning negotiation
0806 18:34:38.660372 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 18:34:38.660782 (+ 410us) server_negotiation.cc:407] Connection header received
0806 18:34:38.661830 (+ 1048us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:34:38.661833 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:34:38.661899 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:34:38.662002 (+ 103us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:34:38.663615 (+ 1613us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:34:38.664162 (+ 547us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:34:38.665898 (+ 1736us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:34:38.666228 (+ 330us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:34:38.666323 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:34:38.669389 (+ 3066us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:34:38.669395 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:34:38.669401 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 18:34:38.669432 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:34:38.671163 (+ 1731us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:34:38.671773 (+ 610us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:34:38.671779 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:34:38.671783 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:34:38.671863 (+ 80us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:34:38.672286 (+ 423us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:34:38.672292 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:34:38.672295 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:34:38.672568 (+ 273us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:34:38.672650 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0806 18:34:38.673350 (+ 700us) server_negotiation.cc:299] Negotiation successful
0806 18:34:38.673501 (+ 151us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
Aug 06 18:34:38 dist-test-slave-1xrh krb5kdc[26549](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041278, etypes {rep=17 tkt=17 ses=17}, oryx/127.19.194.2@KRBTEST.COM for oryx/127.19.194.61@KRBTEST.COM
I20260806 18:34:38.675288 27722 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:34:38.659919 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.194.62:46737 (local address 127.19.194.2:39805)
0806 18:34:38.660606 (+ 687us) negotiation.cc:107] Waiting for socket to connect
0806 18:34:38.660658 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0806 18:34:38.661599 (+ 941us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:34:38.662162 (+ 563us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:34:38.662171 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:34:38.662813 (+ 642us) client_negotiation.cc:189] Negotiated authn=SASL
0806 18:34:38.663446 (+ 633us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:34:38.663454 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:34:38.664325 (+ 871us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:34:38.664327 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:34:38.665778 (+ 1451us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:34:38.665787 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:34:38.666514 (+ 727us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:34:38.666520 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:34:38.666728 (+ 208us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:34:38.667396 (+ 668us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 18:34:38.667421 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0806 18:34:38.669212 (+ 1791us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 18:34:38.671286 (+ 2074us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:34:38.671293 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:34:38.671309 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:34:38.671645 (+ 336us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:34:38.671976 (+ 331us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:34:38.671979 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:34:38.671981 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:34:38.672152 (+ 171us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:34:38.672740 (+ 588us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 18:34:38.672748 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 18:34:38.673051 (+ 303us) client_negotiation.cc:769] Sending connection context
0806 18:34:38.673845 (+ 794us) client_negotiation.cc:240] Negotiation successful
0806 18:34:38.674193 (+ 348us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":503,"mutex_wait_us":174,"spinlock_wait_cycles":22656,"thread_start_us":145,"threads_started":1}
I20260806 18:34:38.677294 27585 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:34:38.661042 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.194.2:35867 (local address 127.19.194.60:41749)
0806 18:34:38.661202 (+ 160us) server_negotiation.cc:206] Beginning negotiation
0806 18:34:38.661207 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 18:34:38.662068 (+ 861us) server_negotiation.cc:407] Connection header received
0806 18:34:38.662236 (+ 168us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:34:38.662241 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:34:38.662296 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:34:38.662386 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:34:38.663580 (+ 1194us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:34:38.664411 (+ 831us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:34:38.668149 (+ 3738us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:34:38.668674 (+ 525us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:34:38.668782 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:34:38.672701 (+ 3919us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:34:38.672708 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:34:38.672714 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 18:34:38.672753 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:34:38.675018 (+ 2265us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:34:38.675720 (+ 702us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:34:38.675727 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:34:38.675732 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:34:38.675847 (+ 115us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:34:38.676177 (+ 330us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:34:38.676183 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:34:38.676188 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:34:38.676430 (+ 242us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:34:38.676521 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0806 18:34:38.676912 (+ 391us) server_negotiation.cc:299] Negotiation successful
0806 18:34:38.677062 (+ 150us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260806 18:34:38.677973 27723 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:34:38.660893 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.194.60:41749 (local address 127.19.194.2:35867)
0806 18:34:38.661989 (+ 1096us) negotiation.cc:107] Waiting for socket to connect
0806 18:34:38.662002 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0806 18:34:38.662127 (+ 125us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:34:38.662416 (+ 289us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:34:38.662419 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:34:38.662798 (+ 379us) client_negotiation.cc:189] Negotiated authn=SASL
0806 18:34:38.663433 (+ 635us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:34:38.663448 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:34:38.665788 (+ 2340us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:34:38.665792 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:34:38.667978 (+ 2186us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:34:38.667994 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:34:38.668980 (+ 986us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:34:38.668986 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:34:38.669123 (+ 137us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:34:38.670003 (+ 880us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 18:34:38.670024 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0806 18:34:38.672494 (+ 2470us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 18:34:38.675191 (+ 2697us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:34:38.675197 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:34:38.675202 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:34:38.675580 (+ 378us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:34:38.675971 (+ 391us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:34:38.675976 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:34:38.675981 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:34:38.676075 (+ 94us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:34:38.676603 (+ 528us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 18:34:38.676609 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 18:34:38.676719 (+ 110us) client_negotiation.cc:769] Sending connection context
0806 18:34:38.677503 (+ 784us) client_negotiation.cc:240] Negotiation successful
0806 18:34:38.677684 (+ 181us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1021,"mutex_wait_us":75,"thread_start_us":82,"threads_started":1}
I20260806 18:34:38.678251 27333 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:34:38.659541 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.194.2:54629 (local address 127.19.194.61:39189)
0806 18:34:38.659721 (+ 180us) server_negotiation.cc:206] Beginning negotiation
0806 18:34:38.659727 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 18:34:38.661261 (+ 1534us) server_negotiation.cc:407] Connection header received
0806 18:34:38.661805 (+ 544us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:34:38.661808 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:34:38.661872 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:34:38.661960 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:34:38.663623 (+ 1663us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:34:38.664411 (+ 788us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:34:38.665991 (+ 1580us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:34:38.666323 (+ 332us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:34:38.666388 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:34:38.674598 (+ 8210us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:34:38.674602 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:34:38.674605 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 18:34:38.674633 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:34:38.676450 (+ 1817us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:34:38.676945 (+ 495us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:34:38.676949 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:34:38.676950 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:34:38.677043 (+ 93us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:34:38.677326 (+ 283us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:34:38.677331 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:34:38.677333 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:34:38.677571 (+ 238us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:34:38.677696 (+ 125us) server_negotiation.cc:1035] Waiting for connection context
0806 18:34:38.677975 (+ 279us) server_negotiation.cc:299] Negotiation successful
0806 18:34:38.678101 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260806 18:34:38.678663 27724 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:34:38.659919 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.194.61:39189 (local address 127.19.194.2:54629)
0806 18:34:38.661174 (+ 1255us) negotiation.cc:107] Waiting for socket to connect
0806 18:34:38.661193 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0806 18:34:38.661616 (+ 423us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:34:38.662162 (+ 546us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:34:38.662172 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:34:38.662861 (+ 689us) client_negotiation.cc:189] Negotiated authn=SASL
0806 18:34:38.663445 (+ 584us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:34:38.663454 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:34:38.664542 (+ 1088us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:34:38.664545 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:34:38.665811 (+ 1266us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:34:38.665820 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:34:38.671990 (+ 6170us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:34:38.671993 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:34:38.672091 (+ 98us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:34:38.672553 (+ 462us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 18:34:38.672566 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0806 18:34:38.674431 (+ 1865us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 18:34:38.676571 (+ 2140us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:34:38.676577 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:34:38.676579 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:34:38.676779 (+ 200us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:34:38.677140 (+ 361us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:34:38.677144 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:34:38.677146 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:34:38.677233 (+ 87us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:34:38.677686 (+ 453us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 18:34:38.677689 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 18:34:38.677788 (+ 99us) client_negotiation.cc:769] Sending connection context
0806 18:34:38.677899 (+ 111us) client_negotiation.cc:240] Negotiation successful
0806 18:34:38.678501 (+ 602us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1158,"mutex_wait_us":308,"spinlock_wait_cycles":54400,"thread_start_us":178,"threads_started":1}
I20260806 18:34:38.679319 27717 heartbeater.cc:347] Connected to a master server at 127.19.194.60:41749
I20260806 18:34:38.679582 27717 heartbeater.cc:464] Registering TS with master...
I20260806 18:34:38.680061 27719 heartbeater.cc:347] Connected to a master server at 127.19.194.61:39189
I20260806 18:34:38.680162 27717 heartbeater.cc:511] Master 127.19.194.60:41749 requested a full tablet report, sending...
I20260806 18:34:38.680173 27719 heartbeater.cc:464] Registering TS with master...
I20260806 18:34:38.680411 27719 heartbeater.cc:511] Master 127.19.194.61:39189 requested a full tablet report, sending...
I20260806 18:34:38.680958 27718 heartbeater.cc:347] Connected to a master server at 127.19.194.62:46737
I20260806 18:34:38.681046 27718 heartbeater.cc:464] Registering TS with master...
I20260806 18:34:38.681238 27718 heartbeater.cc:511] Master 127.19.194.62:46737 requested a full tablet report, sending...
I20260806 18:34:38.681676 27199 ts_manager.cc:194] Registered new tserver with Master: a5630d59c6324f3b87d9d2a1f4024c45 (127.19.194.2:44715)
I20260806 18:34:38.681934 26597 ts_manager.cc:194] Re-registered known tserver with Master: a5630d59c6324f3b87d9d2a1f4024c45 (127.19.194.2:44715)
I20260806 18:34:38.682143 27395 ts_manager.cc:194] Registered new tserver with Master: a5630d59c6324f3b87d9d2a1f4024c45 (127.19.194.2:44715)
I20260806 18:34:38.682452 26597 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.19.194.2@KRBTEST.COM'} at 127.19.194.2:39805
I20260806 18:34:38.684239 20232 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJW69tn/build/debug/bin/kudu
/tmp/dist-test-taskJW69tn/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786041246094603-20232-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786041246094603-20232-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786041246094603-20232-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786041246094603-20232-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.19.194.3:42189
--local_ip_for_outbound_sockets=127.19.194.3
--tserver_master_addrs=127.19.194.62:46737
--webserver_port=46369
--webserver_interface=127.19.194.3
--keytab_file=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786041246094603-20232-0/krb5kdc/oryx_127.19.194.3.keytab
--principal=oryx/127.19.194.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.19.194.20:44897
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.19.194.62:46737,127.19.194.61:39189
--tserver_master_addrs=127.19.194.62:46737,127.19.194.61:39189,127.19.194.60:41749 with env {KRB5CCNAME=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786041246094603-20232-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786041246094603-20232-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786041246094603-20232-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260806 18:34:38.730273 27585 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:34:38.726388 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44178 (local address 127.19.194.60:41749)
0806 18:34:38.726544 (+ 156us) server_negotiation.cc:206] Beginning negotiation
0806 18:34:38.726548 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 18:34:38.726565 (+ 17us) server_negotiation.cc:407] Connection header received
0806 18:34:38.726683 (+ 118us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:34:38.726685 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:34:38.726743 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:34:38.726826 (+ 83us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 18:34:38.727261 (+ 435us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:34:38.727791 (+ 530us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:34:38.729314 (+ 1523us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:34:38.729739 (+ 425us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:34:38.729821 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:34:38.729934 (+ 113us) server_negotiation.cc:1035] Waiting for connection context
0806 18:34:38.730099 (+ 165us) server_negotiation.cc:299] Negotiation successful
0806 18:34:38.730163 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":72}
I20260806 18:34:38.730295 27163 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:34:38.726241 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.194.60:41749 (local address 127.0.0.1:44178)
0806 18:34:38.726397 (+ 156us) negotiation.cc:107] Waiting for socket to connect
0806 18:34:38.726415 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0806 18:34:38.726556 (+ 141us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:34:38.726887 (+ 331us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:34:38.726893 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:34:38.726924 (+ 31us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 18:34:38.727064 (+ 140us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:34:38.727076 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:34:38.727934 (+ 858us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:34:38.727939 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:34:38.729187 (+ 1248us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:34:38.729200 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:34:38.729871 (+ 671us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:34:38.729876 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:34:38.729964 (+ 88us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:34:38.729992 (+ 28us) client_negotiation.cc:769] Sending connection context
0806 18:34:38.730075 (+ 83us) client_negotiation.cc:240] Negotiation successful
0806 18:34:38.730158 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":51}
I20260806 18:34:38.731165 27405 raft_consensus.cc:1203] T 00000000000000000000000000000000 P ad824649dcf140339b33a7ec0f8ebdce [term 1 LEARNER]: Deduplicated request from leader. Original: 1.6->[1.7-1.7] Dedup: 1.7->[]
I20260806 18:34:38.731630 27438 sys_catalog.cc:455] T 00000000000000000000000000000000 P ad824649dcf140339b33a7ec0f8ebdce [sys.catalog]: SysCatalogTable state changed. Reason: New leader a475b841d65e4ef59cb6d53d3e454091. Latest consensus state: current_term: 1 leader_uuid: "a475b841d65e4ef59cb6d53d3e454091" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "a475b841d65e4ef59cb6d53d3e454091" member_type: VOTER last_known_addr { host: "127.19.194.62" port: 46737 } } peers { permanent_uuid: "9a38e245bcc34630ba3cc6fb6ca11cb3" member_type: VOTER last_known_addr { host: "127.19.194.61" port: 39189 } attrs { promote: false } } peers { permanent_uuid: "ad824649dcf140339b33a7ec0f8ebdce" member_type: NON_VOTER last_known_addr { host: "127.19.194.60" port: 41749 } attrs { promote: true } } }
I20260806 18:34:38.731799 27438 sys_catalog.cc:458] T 00000000000000000000000000000000 P ad824649dcf140339b33a7ec0f8ebdce [sys.catalog]: This master's current role is: LEARNER
W20260806 18:34:38.800427 27725 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 18:34:38.800673 27725 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 18:34:38.800752 27725 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 18:34:38.805167 27725 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 18:34:38.805227 27725 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 18:34:38.805326 27725 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.19.194.3
I20260806 18:34:38.810618 27725 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.194.20:44897
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786041246094603-20232-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786041246094603-20232-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786041246094603-20232-0/krb5kdc/oryx_127.19.194.3.keytab
--principal=oryx/127.19.194.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.19.194.3:42189
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786041246094603-20232-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.19.194.3
--webserver_port=46369
--tserver_master_addrs=127.19.194.62:46737,127.19.194.61:39189,127.19.194.60:41749
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27725
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.19.194.3
--log_dir=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786041246094603-20232-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 17:59:13 UTC on ccycloud.quasar-wvsnzx.root.comops.site
build id 4571028
I20260806 18:34:38.811800 27725 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 18:34:38.812995 27725 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 18:34:38.822974 27733 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:34:38.822988 27731 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:34:38.823024 27730 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 18:34:38.823210 27725 server_base.cc:1034] running on GCE node
I20260806 18:34:38.823578 27725 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 18:34:38.824295 27725 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 18:34:38.825503 27725 hybrid_clock.cc:648] HybridClock initialized: now 1786041278825489 us; error 38 us; skew 500 ppm
Aug 06 18:34:38 dist-test-slave-1xrh krb5kdc[26549](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041278, etypes {rep=17 tkt=17 ses=17}, oryx/127.19.194.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 18:34:38.828243 27725 init.cc:377] Logged in from keytab as oryx/127.19.194.3@KRBTEST.COM (short username oryx)
I20260806 18:34:38.829124 27725 webserver.cc:460] Webserver started at http://127.19.194.3:46369/ using document root <none> and password file <none>
I20260806 18:34:38.829676 27725 fs_manager.cc:359] Metadata directory not provided
I20260806 18:34:38.829792 27725 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 18:34:38.833725 27725 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.002s sys 0.000s
I20260806 18:34:38.835719 27740 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 18:34:38.836974 27725 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260806 18:34:38.837121 27725 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786041246094603-20232-0/minicluster-data/ts-2/data,/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786041246094603-20232-0/minicluster-data/ts-2/wal
uuid: "402d0dd8fe1f4b4385b0b94ba213aaa6"
format_stamp: "Formatted at 2026-08-06 18:34:37 on dist-test-slave-1xrh"
server_key: "f3b08bfcc4dc3c1fbe6665eb0d24ecda"
server_key_iv: "deb4eabedf15d2d6658505ee6c07da13"
server_key_version: "encryptionkey@0"
I20260806 18:34:38.837641 27725 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786041246094603-20232-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786041246094603-20232-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786041246094603-20232-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 18:34:38.862069 27725 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 18:34:38.865185 27725 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 18:34:38.865406 27725 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 18:34:38.866043 27725 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 18:34:38.866925 27725 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 18:34:38.866998 27725 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 18:34:38.867079 27725 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 18:34:38.867113 27725 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 18:34:38.876191 27725 rpc_server.cc:307] RPC server started. Bound to: 127.19.194.3:42189
I20260806 18:34:38.876219 27855 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.194.3:42189 every 8 connection(s)
I20260806 18:34:38.877189 27725 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786041246094603-20232-0/minicluster-data/ts-2/data/info.pb
I20260806 18:34:38.881593 20232 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJW69tn/build/debug/bin/kudu as pid 27725
Aug 06 18:34:38 dist-test-slave-1xrh krb5kdc[26549](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041278, etypes {rep=17 tkt=17 ses=17}, oryx/127.19.194.3@KRBTEST.COM for oryx/127.19.194.61@KRBTEST.COM
Aug 06 18:34:38 dist-test-slave-1xrh krb5kdc[26549](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041278, etypes {rep=17 tkt=17 ses=17}, oryx/127.19.194.3@KRBTEST.COM for oryx/127.19.194.60@KRBTEST.COM
Aug 06 18:34:38 dist-test-slave-1xrh krb5kdc[26549](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041278, etypes {rep=17 tkt=17 ses=17}, oryx/127.19.194.3@KRBTEST.COM for oryx/127.19.194.62@KRBTEST.COM
I20260806 18:34:38.897850 27333 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:34:38.880714 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.194.3:48787 (local address 127.19.194.61:39189)
0806 18:34:38.880850 (+ 136us) server_negotiation.cc:206] Beginning negotiation
0806 18:34:38.880855 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 18:34:38.880971 (+ 116us) server_negotiation.cc:407] Connection header received
0806 18:34:38.881503 (+ 532us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:34:38.881507 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:34:38.881563 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:34:38.881619 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:34:38.883600 (+ 1981us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:34:38.884426 (+ 826us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:34:38.887192 (+ 2766us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:34:38.887516 (+ 324us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:34:38.887599 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:34:38.890852 (+ 3253us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:34:38.890857 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:34:38.890861 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 18:34:38.890895 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:34:38.892892 (+ 1997us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:34:38.895918 (+ 3026us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:34:38.895923 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:34:38.895926 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:34:38.896035 (+ 109us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:34:38.896493 (+ 458us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:34:38.896500 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:34:38.896507 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:34:38.896759 (+ 252us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:34:38.896853 (+ 94us) server_negotiation.cc:1035] Waiting for connection context
0806 18:34:38.897553 (+ 700us) server_negotiation.cc:299] Negotiation successful
0806 18:34:38.897678 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260806 18:34:38.899577 27863 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:34:38.880564 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.194.61:39189 (local address 127.19.194.3:48787)
0806 18:34:38.880871 (+ 307us) negotiation.cc:107] Waiting for socket to connect
0806 18:34:38.880885 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0806 18:34:38.881260 (+ 375us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:34:38.882027 (+ 767us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:34:38.882035 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:34:38.882663 (+ 628us) client_negotiation.cc:189] Negotiated authn=SASL
0806 18:34:38.883420 (+ 757us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:34:38.883442 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:34:38.884887 (+ 1445us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:34:38.884891 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:34:38.887029 (+ 2138us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:34:38.887041 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:34:38.887760 (+ 719us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:34:38.887766 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:34:38.887979 (+ 213us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:34:38.888647 (+ 668us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 18:34:38.888669 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0806 18:34:38.890660 (+ 1991us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 18:34:38.894975 (+ 4315us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:34:38.894982 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:34:38.894999 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:34:38.895386 (+ 387us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:34:38.896166 (+ 780us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:34:38.896170 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:34:38.896174 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:34:38.896370 (+ 196us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:34:38.896932 (+ 562us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 18:34:38.896938 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 18:34:38.897262 (+ 324us) client_negotiation.cc:769] Sending connection context
0806 18:34:38.898090 (+ 828us) client_negotiation.cc:240] Negotiation successful
0806 18:34:38.898440 (+ 350us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":249,"mutex_wait_us":454,"thread_start_us":75,"threads_started":1}
I20260806 18:34:38.900817 27862 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:34:38.879483 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.194.60:41749 (local address 127.19.194.3:48081)
0806 18:34:38.880356 (+ 873us) negotiation.cc:107] Waiting for socket to connect
0806 18:34:38.880370 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0806 18:34:38.881258 (+ 888us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:34:38.882043 (+ 785us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:34:38.882048 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:34:38.882750 (+ 702us) client_negotiation.cc:189] Negotiated authn=SASL
0806 18:34:38.883420 (+ 670us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:34:38.883442 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:34:38.885906 (+ 2464us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:34:38.885910 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:34:38.886821 (+ 911us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:34:38.886829 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:34:38.891485 (+ 4656us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:34:38.891489 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:34:38.891589 (+ 100us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:34:38.892033 (+ 444us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 18:34:38.892045 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0806 18:34:38.894493 (+ 2448us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 18:34:38.898662 (+ 4169us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:34:38.898665 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:34:38.898667 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:34:38.898906 (+ 239us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:34:38.899242 (+ 336us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:34:38.899245 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:34:38.899247 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:34:38.899332 (+ 85us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:34:38.899818 (+ 486us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 18:34:38.899821 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 18:34:38.899913 (+ 92us) client_negotiation.cc:769] Sending connection context
0806 18:34:38.900007 (+ 94us) client_negotiation.cc:240] Negotiation successful
0806 18:34:38.900145 (+ 138us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":793,"mutex_wait_us":144,"spinlock_wait_cycles":20352,"thread_start_us":173,"threads_started":1}
I20260806 18:34:38.900822 27871 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:34:38.884252 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44182 (local address 127.19.194.60:41749)
0806 18:34:38.886945 (+ 2693us) server_negotiation.cc:206] Beginning negotiation
0806 18:34:38.886957 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0806 18:34:38.887602 (+ 645us) server_negotiation.cc:407] Connection header received
0806 18:34:38.887643 (+ 41us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:34:38.887646 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:34:38.887697 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:34:38.887777 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:34:38.892654 (+ 4877us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:34:38.893479 (+ 825us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:34:38.894688 (+ 1209us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:34:38.895061 (+ 373us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:34:38.895517 (+ 456us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:34:38.896715 (+ 1198us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:34:38.896718 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:34:38.896721 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 18:34:38.896748 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:34:38.898698 (+ 1950us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:34:38.899209 (+ 511us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:34:38.899212 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:34:38.899214 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:34:38.899293 (+ 79us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:34:38.899555 (+ 262us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:34:38.899558 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:34:38.899559 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:34:38.899891 (+ 332us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:34:38.900009 (+ 118us) server_negotiation.cc:1035] Waiting for connection context
0806 18:34:38.900531 (+ 522us) server_negotiation.cc:299] Negotiation successful
0806 18:34:38.900655 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2614,"spinlock_wait_cycles":8576,"thread_start_us":86,"threads_started":1}
I20260806 18:34:38.901111 27858 heartbeater.cc:347] Connected to a master server at 127.19.194.61:39189
I20260806 18:34:38.901350 27858 heartbeater.cc:464] Registering TS with master...
I20260806 18:34:38.901381 27585 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:34:38.879070 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.194.3:48081 (local address 127.19.194.60:41749)
0806 18:34:38.879225 (+ 155us) server_negotiation.cc:206] Beginning negotiation
0806 18:34:38.879230 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 18:34:38.880445 (+ 1215us) server_negotiation.cc:407] Connection header received
0806 18:34:38.881508 (+ 1063us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:34:38.881513 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:34:38.881593 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:34:38.881656 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:34:38.884884 (+ 3228us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:34:38.885664 (+ 780us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:34:38.886949 (+ 1285us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:34:38.887329 (+ 380us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:34:38.887410 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:34:38.894616 (+ 7206us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:34:38.894623 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:34:38.894628 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 18:34:38.894663 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:34:38.898548 (+ 3885us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:34:38.899020 (+ 472us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:34:38.899023 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:34:38.899025 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:34:38.899111 (+ 86us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:34:38.899433 (+ 322us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:34:38.899436 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:34:38.899438 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:34:38.899671 (+ 233us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:34:38.899759 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0806 18:34:38.901126 (+ 1367us) server_negotiation.cc:299] Negotiation successful
0806 18:34:38.901245 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":73}
I20260806 18:34:38.901935 27858 heartbeater.cc:511] Master 127.19.194.61:39189 requested a full tablet report, sending...
I20260806 18:34:38.902486 27856 heartbeater.cc:347] Connected to a master server at 127.19.194.60:41749
I20260806 18:34:38.902586 27856 heartbeater.cc:464] Registering TS with master...
I20260806 18:34:38.902773 27856 heartbeater.cc:511] Master 127.19.194.60:41749 requested a full tablet report, sending...
I20260806 18:34:38.903723 27199 ts_manager.cc:194] Registered new tserver with Master: 402d0dd8fe1f4b4385b0b94ba213aaa6 (127.19.194.3:42189)
I20260806 18:34:38.904259 26636 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:34:38.879067 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.194.3:44149 (local address 127.19.194.62:46737)
0806 18:34:38.879225 (+ 158us) server_negotiation.cc:206] Beginning negotiation
0806 18:34:38.879230 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 18:34:38.880208 (+ 978us) server_negotiation.cc:407] Connection header received
0806 18:34:38.881503 (+ 1295us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:34:38.881507 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:34:38.881563 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:34:38.881619 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:34:38.883668 (+ 2049us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:34:38.884477 (+ 809us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:34:38.886310 (+ 1833us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:34:38.886530 (+ 220us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:34:38.886642 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:34:38.895541 (+ 8899us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:34:38.895546 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:34:38.895550 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 18:34:38.895582 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:34:38.897301 (+ 1719us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:34:38.902830 (+ 5529us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:34:38.902833 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:34:38.902835 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:34:38.902923 (+ 88us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:34:38.903266 (+ 343us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:34:38.903272 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:34:38.903274 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:34:38.903504 (+ 230us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:34:38.903589 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0806 18:34:38.903981 (+ 392us) server_negotiation.cc:299] Negotiation successful
0806 18:34:38.904108 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":76}
I20260806 18:34:38.904412 27395 ts_manager.cc:194] Registered new tserver with Master: 402d0dd8fe1f4b4385b0b94ba213aaa6 (127.19.194.3:42189)
I20260806 18:34:38.904773 27861 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:34:38.879483 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.194.62:46737 (local address 127.19.194.3:44149)
0806 18:34:38.880022 (+ 539us) negotiation.cc:107] Waiting for socket to connect
0806 18:34:38.880077 (+ 55us) client_negotiation.cc:174] Beginning negotiation
0806 18:34:38.881260 (+ 1183us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:34:38.882027 (+ 767us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:34:38.882035 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:34:38.882674 (+ 639us) client_negotiation.cc:189] Negotiated authn=SASL
0806 18:34:38.883499 (+ 825us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:34:38.883511 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:34:38.884755 (+ 1244us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:34:38.884758 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:34:38.886173 (+ 1415us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:34:38.886183 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:34:38.886694 (+ 511us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:34:38.886699 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:34:38.891809 (+ 5110us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:34:38.892640 (+ 831us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 18:34:38.892654 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0806 18:34:38.894838 (+ 2184us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 18:34:38.901708 (+ 6870us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:34:38.901711 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:34:38.901713 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:34:38.902710 (+ 997us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:34:38.903040 (+ 330us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:34:38.903043 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:34:38.903045 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:34:38.903155 (+ 110us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:34:38.903650 (+ 495us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 18:34:38.903653 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 18:34:38.903738 (+ 85us) client_negotiation.cc:769] Sending connection context
0806 18:34:38.904445 (+ 707us) client_negotiation.cc:240] Negotiation successful
0806 18:34:38.904596 (+ 151us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":331,"mutex_wait_us":63,"spinlock_wait_cycles":9984,"thread_start_us":609,"threads_started":1}
I20260806 18:34:38.905405 27857 heartbeater.cc:347] Connected to a master server at 127.19.194.62:46737
I20260806 18:34:38.905503 27857 heartbeater.cc:464] Registering TS with master...
I20260806 18:34:38.905694 27857 heartbeater.cc:511] Master 127.19.194.62:46737 requested a full tablet report, sending...
I20260806 18:34:38.906334 26597 ts_manager.cc:194] Re-registered known tserver with Master: 402d0dd8fe1f4b4385b0b94ba213aaa6 (127.19.194.3:42189)
I20260806 18:34:38.906711 27873 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:34:38.884613 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37858 (local address 127.19.194.61:39189)
0806 18:34:38.884994 (+ 381us) server_negotiation.cc:206] Beginning negotiation
0806 18:34:38.884998 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 18:34:38.885659 (+ 661us) server_negotiation.cc:407] Connection header received
0806 18:34:38.885698 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:34:38.885701 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:34:38.885746 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:34:38.885824 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:34:38.886815 (+ 991us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:34:38.887602 (+ 787us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:34:38.888825 (+ 1223us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:34:38.889161 (+ 336us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:34:38.889604 (+ 443us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:34:38.890820 (+ 1216us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:34:38.890824 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:34:38.890827 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 18:34:38.890851 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:34:38.894316 (+ 3465us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:34:38.905325 (+ 11009us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:34:38.905328 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:34:38.905330 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:34:38.905417 (+ 87us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:34:38.905787 (+ 370us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:34:38.905793 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:34:38.905795 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:34:38.906022 (+ 227us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:34:38.906147 (+ 125us) server_negotiation.cc:1035] Waiting for connection context
0806 18:34:38.906408 (+ 261us) server_negotiation.cc:299] Negotiation successful
0806 18:34:38.906550 (+ 142us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":4,"server-negotiator.queue_time_us":301,"thread_start_us":109,"threads_started":1}
I20260806 18:34:38.907256 26597 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.19.194.3@KRBTEST.COM'} at 127.19.194.3:44149
I20260806 18:34:38.908007 27872 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:34:38.884584 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44448 (local address 127.19.194.62:46737)
0806 18:34:38.887108 (+ 2524us) server_negotiation.cc:206] Beginning negotiation
0806 18:34:38.887115 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 18:34:38.887150 (+ 35us) server_negotiation.cc:407] Connection header received
0806 18:34:38.887188 (+ 38us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:34:38.887190 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:34:38.887907 (+ 717us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:34:38.889866 (+ 1959us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:34:38.889983 (+ 117us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:34:38.890765 (+ 782us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:34:38.902037 (+ 11272us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:34:38.902358 (+ 321us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:34:38.902481 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:34:38.904007 (+ 1526us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:34:38.904010 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:34:38.904012 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 18:34:38.904042 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:34:38.905838 (+ 1796us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:34:38.906309 (+ 471us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:34:38.906312 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:34:38.906314 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:34:38.906452 (+ 138us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:34:38.906787 (+ 335us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:34:38.906790 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:34:38.906792 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:34:38.907018 (+ 226us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:34:38.907671 (+ 653us) server_negotiation.cc:1035] Waiting for connection context
0806 18:34:38.907746 (+ 75us) server_negotiation.cc:299] Negotiation successful
0806 18:34:38.907871 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2436,"thread_start_us":76,"threads_started":1}
I20260806 18:34:38.910553 26597 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:44448:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260806 18:34:38.913780 26597 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 18:34:38.916278 27359 sys_catalog.cc:455] T 00000000000000000000000000000000 P a475b841d65e4ef59cb6d53d3e454091 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "a475b841d65e4ef59cb6d53d3e454091" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "a475b841d65e4ef59cb6d53d3e454091" member_type: VOTER last_known_addr { host: "127.19.194.62" port: 46737 } } peers { permanent_uuid: "9a38e245bcc34630ba3cc6fb6ca11cb3" member_type: VOTER last_known_addr { host: "127.19.194.61" port: 39189 } attrs { promote: false } } peers { permanent_uuid: "ad824649dcf140339b33a7ec0f8ebdce" member_type: NON_VOTER last_known_addr { host: "127.19.194.60" port: 41749 } attrs { promote: true } } }
I20260806 18:34:38.916415 27359 sys_catalog.cc:458] T 00000000000000000000000000000000 P a475b841d65e4ef59cb6d53d3e454091 [sys.catalog]: This master's current role is: LEADER
I20260806 18:34:38.919920 27359 raft_consensus.cc:1050] T 00000000000000000000000000000000 P a475b841d65e4ef59cb6d53d3e454091: attempting to promote NON_VOTER ad824649dcf140339b33a7ec0f8ebdce to VOTER
I20260806 18:34:38.920250 27359 consensus_queue.cc:237] T 00000000000000000000000000000000 P a475b841d65e4ef59cb6d53d3e454091 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 1.8, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "a475b841d65e4ef59cb6d53d3e454091" member_type: VOTER last_known_addr { host: "127.19.194.62" port: 46737 } } peers { permanent_uuid: "9a38e245bcc34630ba3cc6fb6ca11cb3" member_type: VOTER last_known_addr { host: "127.19.194.61" port: 39189 } attrs { promote: false } } peers { permanent_uuid: "ad824649dcf140339b33a7ec0f8ebdce" member_type: VOTER last_known_addr { host: "127.19.194.60" port: 41749 } attrs { promote: false } }
I20260806 18:34:38.921582 27217 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 9a38e245bcc34630ba3cc6fb6ca11cb3 [term 1 FOLLOWER]: Refusing update from remote peer a475b841d65e4ef59cb6d53d3e454091: Log matching property violated. Preceding OpId in replica: term: 1 index: 8. Preceding OpId from leader: term: 1 index: 9. (index mismatch)
I20260806 18:34:38.922106 27404 raft_consensus.cc:1261] T 00000000000000000000000000000000 P ad824649dcf140339b33a7ec0f8ebdce [term 1 LEARNER]: Refusing update from remote peer a475b841d65e4ef59cb6d53d3e454091: Log matching property violated. Preceding OpId in replica: term: 1 index: 8. Preceding OpId from leader: term: 1 index: 9. (index mismatch)
I20260806 18:34:38.922061 27875 consensus_queue.cc:1035] T 00000000000000000000000000000000 P a475b841d65e4ef59cb6d53d3e454091 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9a38e245bcc34630ba3cc6fb6ca11cb3" member_type: VOTER last_known_addr { host: "127.19.194.61" port: 39189 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260806 18:34:38.923646 27876 consensus_queue.cc:1035] T 00000000000000000000000000000000 P a475b841d65e4ef59cb6d53d3e454091 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ad824649dcf140339b33a7ec0f8ebdce" member_type: VOTER last_known_addr { host: "127.19.194.60" port: 41749 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260806 18:34:38.924813 27875 raft_consensus.cc:2932] T 00000000000000000000000000000000 P a475b841d65e4ef59cb6d53d3e454091 [term 1 LEADER]: Committing config change with OpId 1.9: config changed from index 7 to 9, ad824649dcf140339b33a7ec0f8ebdce (127.19.194.60) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "a475b841d65e4ef59cb6d53d3e454091" member_type: VOTER last_known_addr { host: "127.19.194.62" port: 46737 } } peers { permanent_uuid: "9a38e245bcc34630ba3cc6fb6ca11cb3" member_type: VOTER last_known_addr { host: "127.19.194.61" port: 39189 } attrs { promote: false } } peers { permanent_uuid: "ad824649dcf140339b33a7ec0f8ebdce" member_type: VOTER last_known_addr { host: "127.19.194.60" port: 41749 } attrs { promote: false } } }
I20260806 18:34:38.925319 27217 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 9a38e245bcc34630ba3cc6fb6ca11cb3 [term 1 FOLLOWER]: Committing config change with OpId 1.9: config changed from index 7 to 9, ad824649dcf140339b33a7ec0f8ebdce (127.19.194.60) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "a475b841d65e4ef59cb6d53d3e454091" member_type: VOTER last_known_addr { host: "127.19.194.62" port: 46737 } } peers { permanent_uuid: "9a38e245bcc34630ba3cc6fb6ca11cb3" member_type: VOTER last_known_addr { host: "127.19.194.61" port: 39189 } attrs { promote: false } } peers { permanent_uuid: "ad824649dcf140339b33a7ec0f8ebdce" member_type: VOTER last_known_addr { host: "127.19.194.60" port: 41749 } attrs { promote: false } } }
I20260806 18:34:38.926468 27875 sys_catalog.cc:455] T 00000000000000000000000000000000 P a475b841d65e4ef59cb6d53d3e454091 [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "a475b841d65e4ef59cb6d53d3e454091" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "a475b841d65e4ef59cb6d53d3e454091" member_type: VOTER last_known_addr { host: "127.19.194.62" port: 46737 } } peers { permanent_uuid: "9a38e245bcc34630ba3cc6fb6ca11cb3" member_type: VOTER last_known_addr { host: "127.19.194.61" port: 39189 } attrs { promote: false } } peers { permanent_uuid: "ad824649dcf140339b33a7ec0f8ebdce" member_type: VOTER last_known_addr { host: "127.19.194.60" port: 41749 } attrs { promote: false } } }
I20260806 18:34:38.926502 27877 sys_catalog.cc:455] T 00000000000000000000000000000000 P a475b841d65e4ef59cb6d53d3e454091 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "a475b841d65e4ef59cb6d53d3e454091" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "a475b841d65e4ef59cb6d53d3e454091" member_type: VOTER last_known_addr { host: "127.19.194.62" port: 46737 } } peers { permanent_uuid: "9a38e245bcc34630ba3cc6fb6ca11cb3" member_type: VOTER last_known_addr { host: "127.19.194.61" port: 39189 } attrs { promote: false } } peers { permanent_uuid: "ad824649dcf140339b33a7ec0f8ebdce" member_type: VOTER last_known_addr { host: "127.19.194.60" port: 41749 } attrs { promote: false } } }
I20260806 18:34:38.926579 27875 sys_catalog.cc:458] T 00000000000000000000000000000000 P a475b841d65e4ef59cb6d53d3e454091 [sys.catalog]: This master's current role is: LEADER
I20260806 18:34:38.926602 27877 sys_catalog.cc:458] T 00000000000000000000000000000000 P a475b841d65e4ef59cb6d53d3e454091 [sys.catalog]: This master's current role is: LEADER
I20260806 18:34:38.926759 27404 raft_consensus.cc:2932] T 00000000000000000000000000000000 P ad824649dcf140339b33a7ec0f8ebdce [term 1 FOLLOWER]: Committing config change with OpId 1.9: config changed from index 7 to 9, ad824649dcf140339b33a7ec0f8ebdce (127.19.194.60) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "a475b841d65e4ef59cb6d53d3e454091" member_type: VOTER last_known_addr { host: "127.19.194.62" port: 46737 } } peers { permanent_uuid: "9a38e245bcc34630ba3cc6fb6ca11cb3" member_type: VOTER last_known_addr { host: "127.19.194.61" port: 39189 } attrs { promote: false } } peers { permanent_uuid: "ad824649dcf140339b33a7ec0f8ebdce" member_type: VOTER last_known_addr { host: "127.19.194.60" port: 41749 } attrs { promote: false } } }
I20260806 18:34:38.928529 27438 sys_catalog.cc:455] T 00000000000000000000000000000000 P ad824649dcf140339b33a7ec0f8ebdce [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "a475b841d65e4ef59cb6d53d3e454091" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "a475b841d65e4ef59cb6d53d3e454091" member_type: VOTER last_known_addr { host: "127.19.194.62" port: 46737 } } peers { permanent_uuid: "9a38e245bcc34630ba3cc6fb6ca11cb3" member_type: VOTER last_known_addr { host: "127.19.194.61" port: 39189 } attrs { promote: false } } peers { permanent_uuid: "ad824649dcf140339b33a7ec0f8ebdce" member_type: VOTER last_known_addr { host: "127.19.194.60" port: 41749 } attrs { promote: false } } }
I20260806 18:34:38.928670 27438 sys_catalog.cc:458] T 00000000000000000000000000000000 P ad824649dcf140339b33a7ec0f8ebdce [sys.catalog]: This master's current role is: FOLLOWER
I20260806 18:34:38.929502 27877 sys_catalog.cc:455] T 00000000000000000000000000000000 P a475b841d65e4ef59cb6d53d3e454091 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "a475b841d65e4ef59cb6d53d3e454091" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "a475b841d65e4ef59cb6d53d3e454091" member_type: VOTER last_known_addr { host: "127.19.194.62" port: 46737 } } peers { permanent_uuid: "9a38e245bcc34630ba3cc6fb6ca11cb3" member_type: VOTER last_known_addr { host: "127.19.194.61" port: 39189 } attrs { promote: false } } peers { permanent_uuid: "ad824649dcf140339b33a7ec0f8ebdce" member_type: VOTER last_known_addr { host: "127.19.194.60" port: 41749 } attrs { promote: false } } }
I20260806 18:34:38.929617 27877 sys_catalog.cc:458] T 00000000000000000000000000000000 P a475b841d65e4ef59cb6d53d3e454091 [sys.catalog]: This master's current role is: LEADER
I20260806 18:34:38.929716 27885 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9a38e245bcc34630ba3cc6fb6ca11cb3 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "a475b841d65e4ef59cb6d53d3e454091" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "a475b841d65e4ef59cb6d53d3e454091" member_type: VOTER last_known_addr { host: "127.19.194.62" port: 46737 } } peers { permanent_uuid: "9a38e245bcc34630ba3cc6fb6ca11cb3" member_type: VOTER last_known_addr { host: "127.19.194.61" port: 39189 } attrs { promote: false } } peers { permanent_uuid: "ad824649dcf140339b33a7ec0f8ebdce" member_type: VOTER last_known_addr { host: "127.19.194.60" port: 41749 } attrs { promote: false } } }
I20260806 18:34:38.929857 27885 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9a38e245bcc34630ba3cc6fb6ca11cb3 [sys.catalog]: This master's current role is: FOLLOWER
I20260806 18:34:38.935376 27889 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:34:38.929098 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60484 (local address 127.19.194.3:42189)
0806 18:34:38.930227 (+ 1129us) server_negotiation.cc:206] Beginning negotiation
0806 18:34:38.930239 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0806 18:34:38.930358 (+ 119us) server_negotiation.cc:407] Connection header received
0806 18:34:38.931014 (+ 656us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:34:38.931020 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:34:38.931244 (+ 224us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:34:38.931398 (+ 154us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 18:34:38.931914 (+ 516us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:34:38.932957 (+ 1043us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:34:38.934127 (+ 1170us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:34:38.934779 (+ 652us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:34:38.934904 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:34:38.935072 (+ 168us) server_negotiation.cc:1035] Waiting for connection context
0806 18:34:38.935151 (+ 79us) server_negotiation.cc:299] Negotiation successful
0806 18:34:38.935244 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1008,"thread_start_us":126,"threads_started":1}
I20260806 18:34:38.935364 27887 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:34:38.928899 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.194.3:42189 (local address 127.0.0.1:60484)
0806 18:34:38.930256 (+ 1357us) negotiation.cc:107] Waiting for socket to connect
0806 18:34:38.930267 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0806 18:34:38.930849 (+ 582us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:34:38.931680 (+ 831us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:34:38.931683 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:34:38.931699 (+ 16us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 18:34:38.931788 (+ 89us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:34:38.931794 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:34:38.933070 (+ 1276us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:34:38.933075 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:34:38.933986 (+ 911us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:34:38.933996 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:34:38.934915 (+ 919us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:34:38.934919 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:34:38.935004 (+ 85us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:34:38.935028 (+ 24us) client_negotiation.cc:769] Sending connection context
0806 18:34:38.935094 (+ 66us) client_negotiation.cc:240] Negotiation successful
0806 18:34:38.935220 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1307,"mutex_wait_us":22,"thread_start_us":124,"threads_started":1}
I20260806 18:34:38.935530 27163 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:34:38.928240 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.194.2:44715 (local address 127.0.0.1:45982)
0806 18:34:38.928408 (+ 168us) negotiation.cc:107] Waiting for socket to connect
0806 18:34:38.928425 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0806 18:34:38.928506 (+ 81us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:34:38.931019 (+ 2513us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:34:38.931022 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:34:38.931045 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 18:34:38.931174 (+ 129us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:34:38.931183 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:34:38.932813 (+ 1630us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:34:38.932817 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:34:38.933943 (+ 1126us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:34:38.933953 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:34:38.935052 (+ 1099us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:34:38.935056 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:34:38.935187 (+ 131us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:34:38.935213 (+ 26us) client_negotiation.cc:769] Sending connection context
0806 18:34:38.935263 (+ 50us) client_negotiation.cc:240] Negotiation successful
0806 18:34:38.935329 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":87,"mutex_wait_us":3}
I20260806 18:34:38.936252 27886 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:34:38.928456 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45982 (local address 127.19.194.2:44715)
0806 18:34:38.930431 (+ 1975us) server_negotiation.cc:206] Beginning negotiation
0806 18:34:38.930441 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0806 18:34:38.930466 (+ 25us) server_negotiation.cc:407] Connection header received
0806 18:34:38.930562 (+ 96us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:34:38.930568 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:34:38.930799 (+ 231us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:34:38.931371 (+ 572us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 18:34:38.931533 (+ 162us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:34:38.932694 (+ 1161us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:34:38.934094 (+ 1400us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:34:38.934905 (+ 811us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:34:38.935809 (+ 904us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:34:38.935957 (+ 148us) server_negotiation.cc:1035] Waiting for connection context
0806 18:34:38.936008 (+ 51us) server_negotiation.cc:299] Negotiation successful
0806 18:34:38.936108 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1875,"thread_start_us":115,"threads_started":1}
I20260806 18:34:38.936606 27888 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:34:38.928899 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.194.1:33871 (local address 127.0.0.1:50620)
0806 18:34:38.930639 (+ 1740us) negotiation.cc:107] Waiting for socket to connect
0806 18:34:38.930652 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0806 18:34:38.931566 (+ 914us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:34:38.932046 (+ 480us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:34:38.932049 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:34:38.932068 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 18:34:38.932181 (+ 113us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:34:38.932187 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:34:38.933496 (+ 1309us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:34:38.933500 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:34:38.934597 (+ 1097us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:34:38.934608 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:34:38.936177 (+ 1569us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:34:38.936181 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:34:38.936286 (+ 105us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:34:38.936313 (+ 27us) client_negotiation.cc:769] Sending connection context
0806 18:34:38.936394 (+ 81us) client_negotiation.cc:240] Negotiation successful
0806 18:34:38.936462 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1673,"mutex_wait_us":223,"spinlock_wait_cycles":1837824,"thread_start_us":102,"threads_started":1}
I20260806 18:34:38.936679 27890 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:34:38.929294 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50620 (local address 127.19.194.1:33871)
0806 18:34:38.929591 (+ 297us) server_negotiation.cc:206] Beginning negotiation
0806 18:34:38.929777 (+ 186us) server_negotiation.cc:399] Waiting for connection header
0806 18:34:38.930748 (+ 971us) server_negotiation.cc:407] Connection header received
0806 18:34:38.931703 (+ 955us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:34:38.931709 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:34:38.931900 (+ 191us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:34:38.932063 (+ 163us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 18:34:38.932276 (+ 213us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:34:38.933362 (+ 1086us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:34:38.934741 (+ 1379us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:34:38.935888 (+ 1147us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:34:38.935993 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:34:38.936114 (+ 121us) server_negotiation.cc:1035] Waiting for connection context
0806 18:34:38.936445 (+ 331us) server_negotiation.cc:299] Negotiation successful
0806 18:34:38.936541 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":175,"thread_start_us":75,"threads_started":1}
I20260806 18:34:38.938650 27651 tablet_service.cc:1467] Processing CreateTablet for tablet a9eaceeb88a048169189b05f486c356e (DEFAULT_TABLE table=test-table [id=8f81dcf1f9754464b9fa37ac8444b098]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 18:34:38.938660 27789 tablet_service.cc:1467] Processing CreateTablet for tablet a9eaceeb88a048169189b05f486c356e (DEFAULT_TABLE table=test-table [id=8f81dcf1f9754464b9fa37ac8444b098]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 18:34:38.939052 27511 tablet_service.cc:1467] Processing CreateTablet for tablet a9eaceeb88a048169189b05f486c356e (DEFAULT_TABLE table=test-table [id=8f81dcf1f9754464b9fa37ac8444b098]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 18:34:38.939639 27651 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a9eaceeb88a048169189b05f486c356e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 18:34:38.939729 27789 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a9eaceeb88a048169189b05f486c356e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 18:34:38.939953 27511 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a9eaceeb88a048169189b05f486c356e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 18:34:38.947230 27891 tablet_bootstrap.cc:492] T a9eaceeb88a048169189b05f486c356e P a5630d59c6324f3b87d9d2a1f4024c45: Bootstrap starting.
I20260806 18:34:38.948283 27893 tablet_bootstrap.cc:492] T a9eaceeb88a048169189b05f486c356e P a7dbdd68b24540cda5ada7e1e0d73ccd: Bootstrap starting.
I20260806 18:34:38.949335 27891 tablet_bootstrap.cc:654] T a9eaceeb88a048169189b05f486c356e P a5630d59c6324f3b87d9d2a1f4024c45: Neither blocks nor log segments found. Creating new log.
I20260806 18:34:38.949899 27893 tablet_bootstrap.cc:654] T a9eaceeb88a048169189b05f486c356e P a7dbdd68b24540cda5ada7e1e0d73ccd: Neither blocks nor log segments found. Creating new log.
I20260806 18:34:38.950119 27891 log.cc:824] T a9eaceeb88a048169189b05f486c356e P a5630d59c6324f3b87d9d2a1f4024c45: Log is configured to *not* fsync() on all Append() calls
I20260806 18:34:38.950709 27893 log.cc:824] T a9eaceeb88a048169189b05f486c356e P a7dbdd68b24540cda5ada7e1e0d73ccd: Log is configured to *not* fsync() on all Append() calls
I20260806 18:34:38.951443 27892 tablet_bootstrap.cc:492] T a9eaceeb88a048169189b05f486c356e P 402d0dd8fe1f4b4385b0b94ba213aaa6: Bootstrap starting.
I20260806 18:34:38.952376 27891 tablet_bootstrap.cc:492] T a9eaceeb88a048169189b05f486c356e P a5630d59c6324f3b87d9d2a1f4024c45: No bootstrap required, opened a new log
I20260806 18:34:38.952535 27891 ts_tablet_manager.cc:1397] T a9eaceeb88a048169189b05f486c356e P a5630d59c6324f3b87d9d2a1f4024c45: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260806 18:34:38.953263 27892 tablet_bootstrap.cc:654] T a9eaceeb88a048169189b05f486c356e P 402d0dd8fe1f4b4385b0b94ba213aaa6: Neither blocks nor log segments found. Creating new log.
I20260806 18:34:38.953920 27892 log.cc:824] T a9eaceeb88a048169189b05f486c356e P 402d0dd8fe1f4b4385b0b94ba213aaa6: Log is configured to *not* fsync() on all Append() calls
I20260806 18:34:38.955430 27893 tablet_bootstrap.cc:492] T a9eaceeb88a048169189b05f486c356e P a7dbdd68b24540cda5ada7e1e0d73ccd: No bootstrap required, opened a new log
I20260806 18:34:38.955624 27893 ts_tablet_manager.cc:1397] T a9eaceeb88a048169189b05f486c356e P a7dbdd68b24540cda5ada7e1e0d73ccd: Time spent bootstrapping tablet: real 0.007s user 0.004s sys 0.000s
I20260806 18:34:38.955976 27892 tablet_bootstrap.cc:492] T a9eaceeb88a048169189b05f486c356e P 402d0dd8fe1f4b4385b0b94ba213aaa6: No bootstrap required, opened a new log
I20260806 18:34:38.956159 27892 ts_tablet_manager.cc:1397] T a9eaceeb88a048169189b05f486c356e P 402d0dd8fe1f4b4385b0b94ba213aaa6: Time spent bootstrapping tablet: real 0.005s user 0.004s sys 0.000s
I20260806 18:34:38.956476 27891 raft_consensus.cc:348] T a9eaceeb88a048169189b05f486c356e P a5630d59c6324f3b87d9d2a1f4024c45 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a5630d59c6324f3b87d9d2a1f4024c45" member_type: VOTER last_known_addr { host: "127.19.194.2" port: 44715 } } peers { permanent_uuid: "402d0dd8fe1f4b4385b0b94ba213aaa6" member_type: VOTER last_known_addr { host: "127.19.194.3" port: 42189 } } peers { permanent_uuid: "a7dbdd68b24540cda5ada7e1e0d73ccd" member_type: VOTER last_known_addr { host: "127.19.194.1" port: 33871 } }
I20260806 18:34:38.956772 27891 raft_consensus.cc:374] T a9eaceeb88a048169189b05f486c356e P a5630d59c6324f3b87d9d2a1f4024c45 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 18:34:38.956841 27891 raft_consensus.cc:729] T a9eaceeb88a048169189b05f486c356e P a5630d59c6324f3b87d9d2a1f4024c45 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a5630d59c6324f3b87d9d2a1f4024c45, State: Initialized, Role: FOLLOWER
I20260806 18:34:38.957319 27891 consensus_queue.cc:260] T a9eaceeb88a048169189b05f486c356e P a5630d59c6324f3b87d9d2a1f4024c45 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a5630d59c6324f3b87d9d2a1f4024c45" member_type: VOTER last_known_addr { host: "127.19.194.2" port: 44715 } } peers { permanent_uuid: "402d0dd8fe1f4b4385b0b94ba213aaa6" member_type: VOTER last_known_addr { host: "127.19.194.3" port: 42189 } } peers { permanent_uuid: "a7dbdd68b24540cda5ada7e1e0d73ccd" member_type: VOTER last_known_addr { host: "127.19.194.1" port: 33871 } }
I20260806 18:34:38.958335 27891 ts_tablet_manager.cc:1428] T a9eaceeb88a048169189b05f486c356e P a5630d59c6324f3b87d9d2a1f4024c45: Time spent starting tablet: real 0.006s user 0.006s sys 0.000s
I20260806 18:34:38.958560 27718 heartbeater.cc:503] Master 127.19.194.62:46737 was elected leader, sending a full tablet report...
I20260806 18:34:38.961684 27892 raft_consensus.cc:348] T a9eaceeb88a048169189b05f486c356e P 402d0dd8fe1f4b4385b0b94ba213aaa6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a5630d59c6324f3b87d9d2a1f4024c45" member_type: VOTER last_known_addr { host: "127.19.194.2" port: 44715 } } peers { permanent_uuid: "402d0dd8fe1f4b4385b0b94ba213aaa6" member_type: VOTER last_known_addr { host: "127.19.194.3" port: 42189 } } peers { permanent_uuid: "a7dbdd68b24540cda5ada7e1e0d73ccd" member_type: VOTER last_known_addr { host: "127.19.194.1" port: 33871 } }
I20260806 18:34:38.961966 27892 raft_consensus.cc:374] T a9eaceeb88a048169189b05f486c356e P 402d0dd8fe1f4b4385b0b94ba213aaa6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 18:34:38.962031 27892 raft_consensus.cc:729] T a9eaceeb88a048169189b05f486c356e P 402d0dd8fe1f4b4385b0b94ba213aaa6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 402d0dd8fe1f4b4385b0b94ba213aaa6, State: Initialized, Role: FOLLOWER
I20260806 18:34:38.961853 27893 raft_consensus.cc:348] T a9eaceeb88a048169189b05f486c356e P a7dbdd68b24540cda5ada7e1e0d73ccd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a5630d59c6324f3b87d9d2a1f4024c45" member_type: VOTER last_known_addr { host: "127.19.194.2" port: 44715 } } peers { permanent_uuid: "402d0dd8fe1f4b4385b0b94ba213aaa6" member_type: VOTER last_known_addr { host: "127.19.194.3" port: 42189 } } peers { permanent_uuid: "a7dbdd68b24540cda5ada7e1e0d73ccd" member_type: VOTER last_known_addr { host: "127.19.194.1" port: 33871 } }
I20260806 18:34:38.962126 27893 raft_consensus.cc:374] T a9eaceeb88a048169189b05f486c356e P a7dbdd68b24540cda5ada7e1e0d73ccd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 18:34:38.962186 27893 raft_consensus.cc:729] T a9eaceeb88a048169189b05f486c356e P a7dbdd68b24540cda5ada7e1e0d73ccd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a7dbdd68b24540cda5ada7e1e0d73ccd, State: Initialized, Role: FOLLOWER
I20260806 18:34:38.962466 27892 consensus_queue.cc:260] T a9eaceeb88a048169189b05f486c356e P 402d0dd8fe1f4b4385b0b94ba213aaa6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a5630d59c6324f3b87d9d2a1f4024c45" member_type: VOTER last_known_addr { host: "127.19.194.2" port: 44715 } } peers { permanent_uuid: "402d0dd8fe1f4b4385b0b94ba213aaa6" member_type: VOTER last_known_addr { host: "127.19.194.3" port: 42189 } } peers { permanent_uuid: "a7dbdd68b24540cda5ada7e1e0d73ccd" member_type: VOTER last_known_addr { host: "127.19.194.1" port: 33871 } }
I20260806 18:34:38.962613 27893 consensus_queue.cc:260] T a9eaceeb88a048169189b05f486c356e P a7dbdd68b24540cda5ada7e1e0d73ccd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a5630d59c6324f3b87d9d2a1f4024c45" member_type: VOTER last_known_addr { host: "127.19.194.2" port: 44715 } } peers { permanent_uuid: "402d0dd8fe1f4b4385b0b94ba213aaa6" member_type: VOTER last_known_addr { host: "127.19.194.3" port: 42189 } } peers { permanent_uuid: "a7dbdd68b24540cda5ada7e1e0d73ccd" member_type: VOTER last_known_addr { host: "127.19.194.1" port: 33871 } }
I20260806 18:34:38.963510 27892 ts_tablet_manager.cc:1428] T a9eaceeb88a048169189b05f486c356e P 402d0dd8fe1f4b4385b0b94ba213aaa6: Time spent starting tablet: real 0.007s user 0.007s sys 0.000s
I20260806 18:34:38.963548 27893 ts_tablet_manager.cc:1428] T a9eaceeb88a048169189b05f486c356e P a7dbdd68b24540cda5ada7e1e0d73ccd: Time spent starting tablet: real 0.008s user 0.006s sys 0.000s
I20260806 18:34:38.963742 27579 heartbeater.cc:503] Master 127.19.194.62:46737 was elected leader, sending a full tablet report...
I20260806 18:34:38.963869 27857 heartbeater.cc:503] Master 127.19.194.62:46737 was elected leader, sending a full tablet report...
W20260806 18:34:39.128613 27859 tablet.cc:2369] T a9eaceeb88a048169189b05f486c356e P 402d0dd8fe1f4b4385b0b94ba213aaa6: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 18:34:39.154946 27898 raft_consensus.cc:482] T a9eaceeb88a048169189b05f486c356e P 402d0dd8fe1f4b4385b0b94ba213aaa6 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 18:34:39.155196 27898 raft_consensus.cc:504] T a9eaceeb88a048169189b05f486c356e P 402d0dd8fe1f4b4385b0b94ba213aaa6 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a5630d59c6324f3b87d9d2a1f4024c45" member_type: VOTER last_known_addr { host: "127.19.194.2" port: 44715 } } peers { permanent_uuid: "402d0dd8fe1f4b4385b0b94ba213aaa6" member_type: VOTER last_known_addr { host: "127.19.194.3" port: 42189 } } peers { permanent_uuid: "a7dbdd68b24540cda5ada7e1e0d73ccd" member_type: VOTER last_known_addr { host: "127.19.194.1" port: 33871 } }
I20260806 18:34:39.156400 27898 leader_election.cc:288] T a9eaceeb88a048169189b05f486c356e P 402d0dd8fe1f4b4385b0b94ba213aaa6 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a5630d59c6324f3b87d9d2a1f4024c45 (127.19.194.2:44715), a7dbdd68b24540cda5ada7e1e0d73ccd (127.19.194.1:33871)
W20260806 18:34:39.159754 27721 tablet.cc:2369] T a9eaceeb88a048169189b05f486c356e P a5630d59c6324f3b87d9d2a1f4024c45: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 18:34:39.160436 27861 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:34:39.156719 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.194.1:33871 (local address 127.19.194.3:48677)
0806 18:34:39.157037 (+ 318us) negotiation.cc:107] Waiting for socket to connect
0806 18:34:39.157049 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0806 18:34:39.157159 (+ 110us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:34:39.157427 (+ 268us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:34:39.157430 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:34:39.157446 (+ 16us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 18:34:39.157560 (+ 114us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:34:39.157566 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:34:39.158360 (+ 794us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:34:39.158364 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:34:39.159241 (+ 877us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:34:39.159251 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:34:39.159985 (+ 734us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:34:39.159990 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:34:39.160091 (+ 101us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:34:39.160109 (+ 18us) client_negotiation.cc:769] Sending connection context
0806 18:34:39.160203 (+ 94us) client_negotiation.cc:240] Negotiation successful
0806 18:34:39.160273 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":251}
I20260806 18:34:39.160436 27890 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:34:39.156797 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.194.3:48677 (local address 127.19.194.1:33871)
0806 18:34:39.156954 (+ 157us) server_negotiation.cc:206] Beginning negotiation
0806 18:34:39.156961 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 18:34:39.157117 (+ 156us) server_negotiation.cc:407] Connection header received
0806 18:34:39.157273 (+ 156us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:34:39.157277 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:34:39.157334 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:34:39.157419 (+ 85us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 18:34:39.157672 (+ 253us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:34:39.158233 (+ 561us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:34:39.159364 (+ 1131us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:34:39.159766 (+ 402us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:34:39.159905 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:34:39.159966 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0806 18:34:39.160224 (+ 258us) server_negotiation.cc:299] Negotiation successful
0806 18:34:39.160299 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
I20260806 18:34:39.160450 27863 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:34:39.156640 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.194.2:44715 (local address 127.19.194.3:60893)
0806 18:34:39.156836 (+ 196us) negotiation.cc:107] Waiting for socket to connect
0806 18:34:39.156852 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0806 18:34:39.156964 (+ 112us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:34:39.157263 (+ 299us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:34:39.157266 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:34:39.157283 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 18:34:39.157407 (+ 124us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:34:39.157413 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:34:39.158207 (+ 794us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:34:39.158212 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:34:39.159194 (+ 982us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:34:39.159204 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:34:39.160059 (+ 855us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:34:39.160063 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:34:39.160164 (+ 101us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:34:39.160187 (+ 23us) client_negotiation.cc:769] Sending connection context
0806 18:34:39.160260 (+ 73us) client_negotiation.cc:240] Negotiation successful
0806 18:34:39.160328 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":126}
I20260806 18:34:39.160737 27886 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:34:39.156704 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.194.3:60893 (local address 127.19.194.2:44715)
0806 18:34:39.156849 (+ 145us) server_negotiation.cc:206] Beginning negotiation
0806 18:34:39.156855 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 18:34:39.156931 (+ 76us) server_negotiation.cc:407] Connection header received
0806 18:34:39.157101 (+ 170us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:34:39.157104 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:34:39.157147 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:34:39.157241 (+ 94us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 18:34:39.157521 (+ 280us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:34:39.158096 (+ 575us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:34:39.159314 (+ 1218us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:34:39.159822 (+ 508us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:34:39.159937 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:34:39.159984 (+ 47us) server_negotiation.cc:1035] Waiting for connection context
0806 18:34:39.160521 (+ 537us) server_negotiation.cc:299] Negotiation successful
0806 18:34:39.160606 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260806 18:34:39.161226 27531 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a9eaceeb88a048169189b05f486c356e" candidate_uuid: "402d0dd8fe1f4b4385b0b94ba213aaa6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a7dbdd68b24540cda5ada7e1e0d73ccd" is_pre_election: true
I20260806 18:34:39.161238 27671 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a9eaceeb88a048169189b05f486c356e" candidate_uuid: "402d0dd8fe1f4b4385b0b94ba213aaa6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a5630d59c6324f3b87d9d2a1f4024c45" is_pre_election: true
I20260806 18:34:39.161527 27531 raft_consensus.cc:2454] T a9eaceeb88a048169189b05f486c356e P a7dbdd68b24540cda5ada7e1e0d73ccd [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 402d0dd8fe1f4b4385b0b94ba213aaa6 in term 0.
I20260806 18:34:39.161527 27671 raft_consensus.cc:2454] T a9eaceeb88a048169189b05f486c356e P a5630d59c6324f3b87d9d2a1f4024c45 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 402d0dd8fe1f4b4385b0b94ba213aaa6 in term 0.
I20260806 18:34:39.161991 27744 leader_election.cc:302] T a9eaceeb88a048169189b05f486c356e P 402d0dd8fe1f4b4385b0b94ba213aaa6 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 402d0dd8fe1f4b4385b0b94ba213aaa6, a5630d59c6324f3b87d9d2a1f4024c45; no voters:
I20260806 18:34:39.162307 27898 raft_consensus.cc:2781] T a9eaceeb88a048169189b05f486c356e P 402d0dd8fe1f4b4385b0b94ba213aaa6 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 18:34:39.162457 27898 raft_consensus.cc:482] T a9eaceeb88a048169189b05f486c356e P 402d0dd8fe1f4b4385b0b94ba213aaa6 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 18:34:39.162523 27898 raft_consensus.cc:3037] T a9eaceeb88a048169189b05f486c356e P 402d0dd8fe1f4b4385b0b94ba213aaa6 [term 0 FOLLOWER]: Advancing to term 1
I20260806 18:34:39.163712 27898 raft_consensus.cc:504] T a9eaceeb88a048169189b05f486c356e P 402d0dd8fe1f4b4385b0b94ba213aaa6 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a5630d59c6324f3b87d9d2a1f4024c45" member_type: VOTER last_known_addr { host: "127.19.194.2" port: 44715 } } peers { permanent_uuid: "402d0dd8fe1f4b4385b0b94ba213aaa6" member_type: VOTER last_known_addr { host: "127.19.194.3" port: 42189 } } peers { permanent_uuid: "a7dbdd68b24540cda5ada7e1e0d73ccd" member_type: VOTER last_known_addr { host: "127.19.194.1" port: 33871 } }
I20260806 18:34:39.164069 27898 leader_election.cc:288] T a9eaceeb88a048169189b05f486c356e P 402d0dd8fe1f4b4385b0b94ba213aaa6 [CANDIDATE]: Term 1 election: Requested vote from peers a5630d59c6324f3b87d9d2a1f4024c45 (127.19.194.2:44715), a7dbdd68b24540cda5ada7e1e0d73ccd (127.19.194.1:33871)
I20260806 18:34:39.164387 27671 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a9eaceeb88a048169189b05f486c356e" candidate_uuid: "402d0dd8fe1f4b4385b0b94ba213aaa6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a5630d59c6324f3b87d9d2a1f4024c45"
I20260806 18:34:39.164515 27531 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a9eaceeb88a048169189b05f486c356e" candidate_uuid: "402d0dd8fe1f4b4385b0b94ba213aaa6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a7dbdd68b24540cda5ada7e1e0d73ccd"
I20260806 18:34:39.164515 27671 raft_consensus.cc:3037] T a9eaceeb88a048169189b05f486c356e P a5630d59c6324f3b87d9d2a1f4024c45 [term 0 FOLLOWER]: Advancing to term 1
I20260806 18:34:39.164657 27531 raft_consensus.cc:3037] T a9eaceeb88a048169189b05f486c356e P a7dbdd68b24540cda5ada7e1e0d73ccd [term 0 FOLLOWER]: Advancing to term 1
I20260806 18:34:39.165602 27671 raft_consensus.cc:2454] T a9eaceeb88a048169189b05f486c356e P a5630d59c6324f3b87d9d2a1f4024c45 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 402d0dd8fe1f4b4385b0b94ba213aaa6 in term 1.
I20260806 18:34:39.165771 27531 raft_consensus.cc:2454] T a9eaceeb88a048169189b05f486c356e P a7dbdd68b24540cda5ada7e1e0d73ccd [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 402d0dd8fe1f4b4385b0b94ba213aaa6 in term 1.
I20260806 18:34:39.166097 27741 leader_election.cc:302] T a9eaceeb88a048169189b05f486c356e P 402d0dd8fe1f4b4385b0b94ba213aaa6 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 402d0dd8fe1f4b4385b0b94ba213aaa6, a7dbdd68b24540cda5ada7e1e0d73ccd; no voters:
I20260806 18:34:39.166306 27898 raft_consensus.cc:2781] T a9eaceeb88a048169189b05f486c356e P 402d0dd8fe1f4b4385b0b94ba213aaa6 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 18:34:39.166540 27898 raft_consensus.cc:686] T a9eaceeb88a048169189b05f486c356e P 402d0dd8fe1f4b4385b0b94ba213aaa6 [term 1 LEADER]: Becoming Leader. State: Replica: 402d0dd8fe1f4b4385b0b94ba213aaa6, State: Running, Role: LEADER
I20260806 18:34:39.166872 27898 consensus_queue.cc:237] T a9eaceeb88a048169189b05f486c356e P 402d0dd8fe1f4b4385b0b94ba213aaa6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a5630d59c6324f3b87d9d2a1f4024c45" member_type: VOTER last_known_addr { host: "127.19.194.2" port: 44715 } } peers { permanent_uuid: "402d0dd8fe1f4b4385b0b94ba213aaa6" member_type: VOTER last_known_addr { host: "127.19.194.3" port: 42189 } } peers { permanent_uuid: "a7dbdd68b24540cda5ada7e1e0d73ccd" member_type: VOTER last_known_addr { host: "127.19.194.1" port: 33871 } }
I20260806 18:34:39.170341 26596 catalog_manager.cc:5495] T a9eaceeb88a048169189b05f486c356e P 402d0dd8fe1f4b4385b0b94ba213aaa6 reported cstate change: term changed from 0 to 1, leader changed from <none> to 402d0dd8fe1f4b4385b0b94ba213aaa6 (127.19.194.3). New cstate: current_term: 1 leader_uuid: "402d0dd8fe1f4b4385b0b94ba213aaa6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a5630d59c6324f3b87d9d2a1f4024c45" member_type: VOTER last_known_addr { host: "127.19.194.2" port: 44715 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "402d0dd8fe1f4b4385b0b94ba213aaa6" member_type: VOTER last_known_addr { host: "127.19.194.3" port: 42189 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a7dbdd68b24540cda5ada7e1e0d73ccd" member_type: VOTER last_known_addr { host: "127.19.194.1" port: 33871 } health_report { overall_health: UNKNOWN } } }
W20260806 18:34:39.211119 27581 tablet.cc:2369] T a9eaceeb88a048169189b05f486c356e P a7dbdd68b24540cda5ada7e1e0d73ccd: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 18:34:39.227918 27889 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:34:39.224488 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60488 (local address 127.19.194.3:42189)
0806 18:34:39.224618 (+ 130us) server_negotiation.cc:206] Beginning negotiation
0806 18:34:39.224625 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 18:34:39.224643 (+ 18us) server_negotiation.cc:407] Connection header received
0806 18:34:39.224798 (+ 155us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:34:39.224801 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:34:39.224878 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:34:39.224975 (+ 97us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 18:34:39.225244 (+ 269us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:34:39.225780 (+ 536us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:34:39.226608 (+ 828us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:34:39.226809 (+ 201us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:34:39.226885 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:34:39.227206 (+ 321us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 18:34:39.227606 (+ 400us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 18:34:39.227700 (+ 94us) server_negotiation.cc:1035] Waiting for connection context
0806 18:34:39.227759 (+ 59us) server_negotiation.cc:299] Negotiation successful
0806 18:34:39.227810 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260806 18:34:39.233955 27671 raft_consensus.cc:1261] T a9eaceeb88a048169189b05f486c356e P a5630d59c6324f3b87d9d2a1f4024c45 [term 1 FOLLOWER]: Refusing update from remote peer 402d0dd8fe1f4b4385b0b94ba213aaa6: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 18:34:39.233956 27531 raft_consensus.cc:1261] T a9eaceeb88a048169189b05f486c356e P a7dbdd68b24540cda5ada7e1e0d73ccd [term 1 FOLLOWER]: Refusing update from remote peer 402d0dd8fe1f4b4385b0b94ba213aaa6: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 18:34:39.234699 27898 consensus_queue.cc:1035] T a9eaceeb88a048169189b05f486c356e P 402d0dd8fe1f4b4385b0b94ba213aaa6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a5630d59c6324f3b87d9d2a1f4024c45" member_type: VOTER last_known_addr { host: "127.19.194.2" port: 44715 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 18:34:39.234900 27900 consensus_queue.cc:1035] T a9eaceeb88a048169189b05f486c356e P 402d0dd8fe1f4b4385b0b94ba213aaa6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a7dbdd68b24540cda5ada7e1e0d73ccd" member_type: VOTER last_known_addr { host: "127.19.194.1" port: 33871 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 18:34:39.247500 27904 mvcc.cc:205] Tried to move back new op lower bound from 7315625079734648832 to 7315625079470391296. Current Snapshot: MvccSnapshot[applied={T|T < 7315625079734648832}]
I20260806 18:34:39.254801 26596 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:44448:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 18:34:39.254989 26596 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:44448:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 18:34:39.259989 26596 catalog_manager.cc:5782] T 00000000000000000000000000000000 P a475b841d65e4ef59cb6d53d3e454091: Sending DeleteTablet for 3 replicas of tablet a9eaceeb88a048169189b05f486c356e
I20260806 18:34:39.260751 27511 tablet_service.cc:1514] Processing DeleteTablet for tablet a9eaceeb88a048169189b05f486c356e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 18:34:39 UTC) from {username='oryx', principal='oryx/127.19.194.62@KRBTEST.COM'} at 127.0.0.1:50620
I20260806 18:34:39.260754 27789 tablet_service.cc:1514] Processing DeleteTablet for tablet a9eaceeb88a048169189b05f486c356e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 18:34:39 UTC) from {username='oryx', principal='oryx/127.19.194.62@KRBTEST.COM'} at 127.0.0.1:60484
I20260806 18:34:39.261366 27651 tablet_service.cc:1514] Processing DeleteTablet for tablet a9eaceeb88a048169189b05f486c356e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 18:34:39 UTC) from {username='oryx', principal='oryx/127.19.194.62@KRBTEST.COM'} at 127.0.0.1:45982
I20260806 18:34:39.261448 27914 tablet_replica.cc:330] T a9eaceeb88a048169189b05f486c356e P 402d0dd8fe1f4b4385b0b94ba213aaa6: stopping tablet replica
I20260806 18:34:39.261763 27915 tablet_replica.cc:330] T a9eaceeb88a048169189b05f486c356e P a5630d59c6324f3b87d9d2a1f4024c45: stopping tablet replica
I20260806 18:34:39.261801 27914 raft_consensus.cc:2229] T a9eaceeb88a048169189b05f486c356e P 402d0dd8fe1f4b4385b0b94ba213aaa6 [term 1 LEADER]: Raft consensus shutting down.
I20260806 18:34:39.262054 27915 raft_consensus.cc:2229] T a9eaceeb88a048169189b05f486c356e P a5630d59c6324f3b87d9d2a1f4024c45 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 18:34:39.262360 27915 raft_consensus.cc:2258] T a9eaceeb88a048169189b05f486c356e P a5630d59c6324f3b87d9d2a1f4024c45 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 18:34:39.262542 27914 raft_consensus.cc:2258] T a9eaceeb88a048169189b05f486c356e P 402d0dd8fe1f4b4385b0b94ba213aaa6 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 18:34:39.262903 27913 tablet_replica.cc:330] T a9eaceeb88a048169189b05f486c356e P a7dbdd68b24540cda5ada7e1e0d73ccd: stopping tablet replica
I20260806 18:34:39.263795 20232 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJW69tn/build/debug/bin/kudu with pid 26563
I20260806 18:34:39.265172 27914 ts_tablet_manager.cc:1905] T a9eaceeb88a048169189b05f486c356e P 402d0dd8fe1f4b4385b0b94ba213aaa6: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 18:34:39.265472 27913 raft_consensus.cc:2229] T a9eaceeb88a048169189b05f486c356e P a7dbdd68b24540cda5ada7e1e0d73ccd [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 18:34:39.265790 27915 ts_tablet_manager.cc:1905] T a9eaceeb88a048169189b05f486c356e P a5630d59c6324f3b87d9d2a1f4024c45: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 18:34:39.268527 27915 ts_tablet_manager.cc:1918] T a9eaceeb88a048169189b05f486c356e P a5630d59c6324f3b87d9d2a1f4024c45: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 18:34:39.268536 27914 ts_tablet_manager.cc:1918] T a9eaceeb88a048169189b05f486c356e P 402d0dd8fe1f4b4385b0b94ba213aaa6: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 18:34:39.265832 27913 raft_consensus.cc:2258] T a9eaceeb88a048169189b05f486c356e P a7dbdd68b24540cda5ada7e1e0d73ccd [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 18:34:39.268639 27915 log.cc:1196] T a9eaceeb88a048169189b05f486c356e P a5630d59c6324f3b87d9d2a1f4024c45: Deleting WAL directory at /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786041246094603-20232-0/minicluster-data/ts-1/wal/wals/a9eaceeb88a048169189b05f486c356e
I20260806 18:34:39.268990 27915 ts_tablet_manager.cc:1939] T a9eaceeb88a048169189b05f486c356e P a5630d59c6324f3b87d9d2a1f4024c45: Deleting consensus metadata
I20260806 18:34:39.270222 27913 ts_tablet_manager.cc:1905] T a9eaceeb88a048169189b05f486c356e P a7dbdd68b24540cda5ada7e1e0d73ccd: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 18:34:39.273156 27913 ts_tablet_manager.cc:1918] T a9eaceeb88a048169189b05f486c356e P a7dbdd68b24540cda5ada7e1e0d73ccd: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 18:34:39.273301 27914 log.cc:1196] T a9eaceeb88a048169189b05f486c356e P 402d0dd8fe1f4b4385b0b94ba213aaa6: Deleting WAL directory at /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786041246094603-20232-0/minicluster-data/ts-2/wal/wals/a9eaceeb88a048169189b05f486c356e
I20260806 18:34:39.273743 27914 ts_tablet_manager.cc:1939] T a9eaceeb88a048169189b05f486c356e P 402d0dd8fe1f4b4385b0b94ba213aaa6: Deleting consensus metadata
W20260806 18:34:39.274677 27603 connection.cc:535] server connection from 127.0.0.1:45982 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20260806 18:34:39.274823 27743 connection.cc:441] server connection from 127.0.0.1:60484 torn down before Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:60484 (request call id 1) could send its response
I20260806 18:34:39.275079 27913 log.cc:1196] T a9eaceeb88a048169189b05f486c356e P a7dbdd68b24540cda5ada7e1e0d73ccd: Deleting WAL directory at /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786041246094603-20232-0/minicluster-data/ts-0/wal/wals/a9eaceeb88a048169189b05f486c356e
I20260806 18:34:39.275472 27913 ts_tablet_manager.cc:1939] T a9eaceeb88a048169189b05f486c356e P a7dbdd68b24540cda5ada7e1e0d73ccd: Deleting consensus metadata
W20260806 18:34:39.276628 27466 connection.cc:441] server connection from 127.0.0.1:50620 torn down before Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:50620 (request call id 1) could send its response
I20260806 18:34:39.281385 27873 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:34:39.262284 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37860 (local address 127.19.194.61:39189)
0806 18:34:39.262390 (+ 106us) server_negotiation.cc:206] Beginning negotiation
0806 18:34:39.262394 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 18:34:39.262802 (+ 408us) server_negotiation.cc:407] Connection header received
0806 18:34:39.270385 (+ 7583us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:34:39.270388 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:34:39.270444 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:34:39.270496 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:34:39.271593 (+ 1097us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:34:39.272399 (+ 806us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:34:39.273665 (+ 1266us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:34:39.274045 (+ 380us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:34:39.274127 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:34:39.277757 (+ 3630us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:34:39.277761 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:34:39.277764 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 18:34:39.277792 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:34:39.279450 (+ 1658us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:34:39.279957 (+ 507us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:34:39.279960 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:34:39.279962 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:34:39.280046 (+ 84us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:34:39.280363 (+ 317us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:34:39.280367 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:34:39.280370 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:34:39.280624 (+ 254us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:34:39.280784 (+ 160us) server_negotiation.cc:1035] Waiting for connection context
0806 18:34:39.281071 (+ 287us) server_negotiation.cc:299] Negotiation successful
0806 18:34:39.281226 (+ 155us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260806 18:34:39.289162 27333 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:34:39.279520 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37874 (local address 127.19.194.61:39189)
0806 18:34:39.279675 (+ 155us) server_negotiation.cc:206] Beginning negotiation
0806 18:34:39.279680 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 18:34:39.279841 (+ 161us) server_negotiation.cc:407] Connection header received
0806 18:34:39.280144 (+ 303us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:34:39.280147 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:34:39.280198 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:34:39.280240 (+ 42us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:34:39.281116 (+ 876us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:34:39.281783 (+ 667us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:34:39.282959 (+ 1176us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:34:39.283356 (+ 397us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:34:39.283458 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:34:39.285196 (+ 1738us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:34:39.285200 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:34:39.285203 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 18:34:39.285228 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:34:39.287067 (+ 1839us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:34:39.287727 (+ 660us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:34:39.287730 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:34:39.287732 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:34:39.287812 (+ 80us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:34:39.288095 (+ 283us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:34:39.288098 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:34:39.288100 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:34:39.288298 (+ 198us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:34:39.288412 (+ 114us) server_negotiation.cc:1035] Waiting for connection context
0806 18:34:39.288910 (+ 498us) server_negotiation.cc:299] Negotiation successful
0806 18:34:39.289011 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260806 18:34:39.291563 27585 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:34:39.262061 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44198 (local address 127.19.194.60:41749)
0806 18:34:39.262192 (+ 131us) server_negotiation.cc:206] Beginning negotiation
0806 18:34:39.262197 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 18:34:39.281410 (+ 19213us) server_negotiation.cc:407] Connection header received
0806 18:34:39.281643 (+ 233us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:34:39.281648 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:34:39.281701 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:34:39.281755 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:34:39.282804 (+ 1049us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:34:39.283606 (+ 802us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:34:39.284819 (+ 1213us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:34:39.285203 (+ 384us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:34:39.285303 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:34:39.287358 (+ 2055us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:34:39.287361 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:34:39.287364 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 18:34:39.287388 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:34:39.289441 (+ 2053us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:34:39.289969 (+ 528us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:34:39.289976 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:34:39.289978 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:34:39.290081 (+ 103us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:34:39.290390 (+ 309us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:34:39.290397 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:34:39.290401 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:34:39.290623 (+ 222us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:34:39.290756 (+ 133us) server_negotiation.cc:1035] Waiting for connection context
0806 18:34:39.291190 (+ 434us) server_negotiation.cc:299] Negotiation successful
0806 18:34:39.291452 (+ 262us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260806 18:34:39.293658 27871 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:34:39.279801 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44212 (local address 127.19.194.60:41749)
0806 18:34:39.279941 (+ 140us) server_negotiation.cc:206] Beginning negotiation
0806 18:34:39.279947 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 18:34:39.283180 (+ 3233us) server_negotiation.cc:407] Connection header received
0806 18:34:39.283343 (+ 163us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:34:39.283346 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:34:39.283383 (+ 37us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:34:39.283430 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:34:39.285911 (+ 2481us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:34:39.286669 (+ 758us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:34:39.287897 (+ 1228us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:34:39.288287 (+ 390us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:34:39.288377 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:34:39.290163 (+ 1786us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:34:39.290166 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:34:39.290169 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 18:34:39.290193 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:34:39.291916 (+ 1723us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:34:39.292452 (+ 536us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:34:39.292454 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:34:39.292455 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:34:39.292532 (+ 77us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:34:39.292812 (+ 280us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:34:39.292816 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:34:39.292818 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:34:39.292977 (+ 159us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:34:39.293107 (+ 130us) server_negotiation.cc:1035] Waiting for connection context
0806 18:34:39.293339 (+ 232us) server_negotiation.cc:299] Negotiation successful
0806 18:34:39.293480 (+ 141us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260806 18:34:40.245843 27923 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:34:40.245325 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.194.62:46737 (local address 127.19.194.1:44841)
0806 18:34:40.245656 (+ 331us) negotiation.cc:107] Waiting for socket to connect
0806 18:34:40.245745 (+ 89us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.194.62:46737: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":253,"thread_start_us":101,"threads_started":1}
W20260806 18:34:40.246280 27579 heartbeater.cc:644] Failed to heartbeat to 127.19.194.62:46737 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.19.194.62:46737: connect: Connection refused (error 111)
I20260806 18:34:40.248076 27924 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:34:40.247557 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.194.62:46737 (local address 127.19.194.2:46201)
0806 18:34:40.247872 (+ 315us) negotiation.cc:107] Waiting for socket to connect
0806 18:34:40.247966 (+ 94us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.194.62:46737: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":231,"thread_start_us":70,"threads_started":1}
I20260806 18:34:40.248215 27925 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:34:40.247689 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.194.62:46737 (local address 127.19.194.3:33093)
0806 18:34:40.248022 (+ 333us) negotiation.cc:107] Waiting for socket to connect
0806 18:34:40.248116 (+ 94us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.194.62:46737: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":261,"thread_start_us":60,"threads_started":1}
W20260806 18:34:40.248562 27857 heartbeater.cc:644] Failed to heartbeat to 127.19.194.62:46737 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.19.194.62:46737: connect: Connection refused (error 111)
W20260806 18:34:40.248765 27718 heartbeater.cc:644] Failed to heartbeat to 127.19.194.62:46737 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.19.194.62:46737: connect: Connection refused (error 111)
I20260806 18:34:40.760586 27927 raft_consensus.cc:482] T 00000000000000000000000000000000 P 9a38e245bcc34630ba3cc6fb6ca11cb3 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader a475b841d65e4ef59cb6d53d3e454091)
I20260806 18:34:40.760614 27926 raft_consensus.cc:482] T 00000000000000000000000000000000 P ad824649dcf140339b33a7ec0f8ebdce [term 1 FOLLOWER]: Starting pre-election (detected failure of leader a475b841d65e4ef59cb6d53d3e454091)
I20260806 18:34:40.760715 27927 raft_consensus.cc:504] T 00000000000000000000000000000000 P 9a38e245bcc34630ba3cc6fb6ca11cb3 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "a475b841d65e4ef59cb6d53d3e454091" member_type: VOTER last_known_addr { host: "127.19.194.62" port: 46737 } } peers { permanent_uuid: "9a38e245bcc34630ba3cc6fb6ca11cb3" member_type: VOTER last_known_addr { host: "127.19.194.61" port: 39189 } attrs { promote: false } } peers { permanent_uuid: "ad824649dcf140339b33a7ec0f8ebdce" member_type: VOTER last_known_addr { host: "127.19.194.60" port: 41749 } attrs { promote: false } }
I20260806 18:34:40.760849 27926 raft_consensus.cc:504] T 00000000000000000000000000000000 P ad824649dcf140339b33a7ec0f8ebdce [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "a475b841d65e4ef59cb6d53d3e454091" member_type: VOTER last_known_addr { host: "127.19.194.62" port: 46737 } } peers { permanent_uuid: "9a38e245bcc34630ba3cc6fb6ca11cb3" member_type: VOTER last_known_addr { host: "127.19.194.61" port: 39189 } attrs { promote: false } } peers { permanent_uuid: "ad824649dcf140339b33a7ec0f8ebdce" member_type: VOTER last_known_addr { host: "127.19.194.60" port: 41749 } attrs { promote: false } }
I20260806 18:34:40.761286 27927 leader_election.cc:288] T 00000000000000000000000000000000 P 9a38e245bcc34630ba3cc6fb6ca11cb3 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers a475b841d65e4ef59cb6d53d3e454091 (127.19.194.62:46737), ad824649dcf140339b33a7ec0f8ebdce (127.19.194.60:41749)
I20260806 18:34:40.762041 27928 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:34:40.761431 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.194.62:46737 (local address 127.0.0.1:45970)
0806 18:34:40.761806 (+ 375us) negotiation.cc:107] Waiting for socket to connect
0806 18:34:40.761907 (+ 101us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.194.62:46737: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":295,"thread_start_us":72,"threads_started":1}
I20260806 18:34:40.762017 27926 leader_election.cc:288] T 00000000000000000000000000000000 P ad824649dcf140339b33a7ec0f8ebdce [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers a475b841d65e4ef59cb6d53d3e454091 (127.19.194.62:46737), 9a38e245bcc34630ba3cc6fb6ca11cb3 (127.19.194.61:39189)
W20260806 18:34:40.762286 27194 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.19.194.62:46737: connect: Connection refused (error 111)
I20260806 18:34:40.762956 27934 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:34:40.762560 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.194.62:46737 (local address 127.0.0.1:45980)
0806 18:34:40.762774 (+ 214us) negotiation.cc:107] Waiting for socket to connect
0806 18:34:40.762864 (+ 90us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.194.62:46737: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":144,"thread_start_us":54,"threads_started":1}
I20260806 18:34:40.763302 27928 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:34:40.763076 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.194.62:46737 (local address 127.0.0.1:45996)
0806 18:34:40.763205 (+ 129us) negotiation.cc:107] Waiting for socket to connect
0806 18:34:40.763252 (+ 47us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.194.62:46737: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":69}
W20260806 18:34:40.763485 27194 leader_election.cc:334] T 00000000000000000000000000000000 P 9a38e245bcc34630ba3cc6fb6ca11cb3 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer a475b841d65e4ef59cb6d53d3e454091 (127.19.194.62:46737): Network error: Client connection negotiation failed: client connection to 127.19.194.62:46737: connect: Connection refused (error 111)
W20260806 18:34:40.763538 27382 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.19.194.62:46737: connect: Connection refused (error 111)
I20260806 18:34:40.765640 27934 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:34:40.765394 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.194.62:46737 (local address 127.0.0.1:46002)
0806 18:34:40.765534 (+ 140us) negotiation.cc:107] Waiting for socket to connect
0806 18:34:40.765584 (+ 50us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.194.62:46737: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":67}
W20260806 18:34:40.765781 27382 leader_election.cc:334] T 00000000000000000000000000000000 P ad824649dcf140339b33a7ec0f8ebdce [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer a475b841d65e4ef59cb6d53d3e454091 (127.19.194.62:46737): Network error: Client connection negotiation failed: client connection to 127.19.194.62:46737: connect: Connection refused (error 111)
I20260806 18:34:40.766481 27929 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:34:40.761496 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.194.60:41749 (local address 127.0.0.1:53832)
0806 18:34:40.761887 (+ 391us) negotiation.cc:107] Waiting for socket to connect
0806 18:34:40.761912 (+ 25us) client_negotiation.cc:174] Beginning negotiation
0806 18:34:40.762010 (+ 98us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:34:40.762581 (+ 571us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:34:40.762587 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:34:40.762609 (+ 22us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 18:34:40.762733 (+ 124us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:34:40.762741 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:34:40.763856 (+ 1115us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:34:40.763861 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:34:40.765133 (+ 1272us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:34:40.765146 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:34:40.766011 (+ 865us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:34:40.766015 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:34:40.766162 (+ 147us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:34:40.766182 (+ 20us) client_negotiation.cc:769] Sending connection context
0806 18:34:40.766278 (+ 96us) client_negotiation.cc:240] Negotiation successful
0806 18:34:40.766353 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":333,"mutex_wait_us":49,"thread_start_us":63,"threads_started":1}
I20260806 18:34:40.766489 27930 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:34:40.761678 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53832 (local address 127.19.194.60:41749)
0806 18:34:40.762208 (+ 530us) server_negotiation.cc:206] Beginning negotiation
0806 18:34:40.762215 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 18:34:40.762228 (+ 13us) server_negotiation.cc:407] Connection header received
0806 18:34:40.762275 (+ 47us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:34:40.762278 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:34:40.762327 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:34:40.762416 (+ 89us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 18:34:40.762891 (+ 475us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:34:40.763708 (+ 817us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:34:40.765263 (+ 1555us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:34:40.765845 (+ 582us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:34:40.765973 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:34:40.766037 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0806 18:34:40.766298 (+ 261us) server_negotiation.cc:299] Negotiation successful
0806 18:34:40.766372 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":3,"server-negotiator.queue_time_us":454,"thread_start_us":70,"threads_started":1}
I20260806 18:34:40.766690 27932 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:34:40.762260 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48498 (local address 127.19.194.61:39189)
0806 18:34:40.762827 (+ 567us) server_negotiation.cc:206] Beginning negotiation
0806 18:34:40.762831 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 18:34:40.762842 (+ 11us) server_negotiation.cc:407] Connection header received
0806 18:34:40.762879 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:34:40.762882 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:34:40.762919 (+ 37us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:34:40.763262 (+ 343us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 18:34:40.763348 (+ 86us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:34:40.764090 (+ 742us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:34:40.765532 (+ 1442us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:34:40.766112 (+ 580us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:34:40.766232 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:34:40.766287 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0806 18:34:40.766472 (+ 185us) server_negotiation.cc:299] Negotiation successful
0806 18:34:40.766543 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":1,"server-negotiator.queue_time_us":478,"thread_start_us":68,"threads_started":1}
I20260806 18:34:40.766913 27404 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "9a38e245bcc34630ba3cc6fb6ca11cb3" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "ad824649dcf140339b33a7ec0f8ebdce" is_pre_election: true
I20260806 18:34:40.767122 27931 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:34:40.762160 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.194.61:39189 (local address 127.0.0.1:48498)
0806 18:34:40.762535 (+ 375us) negotiation.cc:107] Waiting for socket to connect
0806 18:34:40.762552 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0806 18:34:40.762621 (+ 69us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:34:40.763041 (+ 420us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:34:40.763044 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:34:40.763062 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 18:34:40.763161 (+ 99us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:34:40.763168 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:34:40.764213 (+ 1045us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:34:40.764216 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:34:40.765408 (+ 1192us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:34:40.765421 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:34:40.766235 (+ 814us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:34:40.766239 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:34:40.766335 (+ 96us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:34:40.766362 (+ 27us) client_negotiation.cc:769] Sending connection context
0806 18:34:40.766454 (+ 92us) client_negotiation.cc:240] Negotiation successful
0806 18:34:40.766978 (+ 524us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":294,"thread_start_us":81,"threads_started":1}
I20260806 18:34:40.767167 27404 raft_consensus.cc:2454] T 00000000000000000000000000000000 P ad824649dcf140339b33a7ec0f8ebdce [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9a38e245bcc34630ba3cc6fb6ca11cb3 in term 1.
I20260806 18:34:40.767560 27191 leader_election.cc:302] T 00000000000000000000000000000000 P 9a38e245bcc34630ba3cc6fb6ca11cb3 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9a38e245bcc34630ba3cc6fb6ca11cb3, ad824649dcf140339b33a7ec0f8ebdce; no voters: a475b841d65e4ef59cb6d53d3e454091
I20260806 18:34:40.767592 27217 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "ad824649dcf140339b33a7ec0f8ebdce" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "9a38e245bcc34630ba3cc6fb6ca11cb3" is_pre_election: true
I20260806 18:34:40.767740 27217 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 9a38e245bcc34630ba3cc6fb6ca11cb3 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ad824649dcf140339b33a7ec0f8ebdce in term 1.
I20260806 18:34:40.767956 27927 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 9a38e245bcc34630ba3cc6fb6ca11cb3 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260806 18:34:40.768014 27927 raft_consensus.cc:482] T 00000000000000000000000000000000 P 9a38e245bcc34630ba3cc6fb6ca11cb3 [term 1 FOLLOWER]: Starting leader election (detected failure of leader a475b841d65e4ef59cb6d53d3e454091)
I20260806 18:34:40.768065 27927 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 9a38e245bcc34630ba3cc6fb6ca11cb3 [term 1 FOLLOWER]: Advancing to term 2
I20260806 18:34:40.768086 27379 leader_election.cc:302] T 00000000000000000000000000000000 P ad824649dcf140339b33a7ec0f8ebdce [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9a38e245bcc34630ba3cc6fb6ca11cb3, ad824649dcf140339b33a7ec0f8ebdce; no voters: a475b841d65e4ef59cb6d53d3e454091
I20260806 18:34:40.768321 27926 raft_consensus.cc:2781] T 00000000000000000000000000000000 P ad824649dcf140339b33a7ec0f8ebdce [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260806 18:34:40.768375 27926 raft_consensus.cc:482] T 00000000000000000000000000000000 P ad824649dcf140339b33a7ec0f8ebdce [term 1 FOLLOWER]: Starting leader election (detected failure of leader a475b841d65e4ef59cb6d53d3e454091)
I20260806 18:34:40.768415 27926 raft_consensus.cc:3037] T 00000000000000000000000000000000 P ad824649dcf140339b33a7ec0f8ebdce [term 1 FOLLOWER]: Advancing to term 2
I20260806 18:34:40.769129 27927 raft_consensus.cc:504] T 00000000000000000000000000000000 P 9a38e245bcc34630ba3cc6fb6ca11cb3 [term 2 FOLLOWER]: Starting leader election with config: opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "a475b841d65e4ef59cb6d53d3e454091" member_type: VOTER last_known_addr { host: "127.19.194.62" port: 46737 } } peers { permanent_uuid: "9a38e245bcc34630ba3cc6fb6ca11cb3" member_type: VOTER last_known_addr { host: "127.19.194.61" port: 39189 } attrs { promote: false } } peers { permanent_uuid: "ad824649dcf140339b33a7ec0f8ebdce" member_type: VOTER last_known_addr { host: "127.19.194.60" port: 41749 } attrs { promote: false } }
I20260806 18:34:40.769421 27927 leader_election.cc:288] T 00000000000000000000000000000000 P 9a38e245bcc34630ba3cc6fb6ca11cb3 [CANDIDATE]: Term 2 election: Requested vote from peers a475b841d65e4ef59cb6d53d3e454091 (127.19.194.62:46737), ad824649dcf140339b33a7ec0f8ebdce (127.19.194.60:41749)
I20260806 18:34:40.769431 27926 raft_consensus.cc:504] T 00000000000000000000000000000000 P ad824649dcf140339b33a7ec0f8ebdce [term 2 FOLLOWER]: Starting leader election with config: opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "a475b841d65e4ef59cb6d53d3e454091" member_type: VOTER last_known_addr { host: "127.19.194.62" port: 46737 } } peers { permanent_uuid: "9a38e245bcc34630ba3cc6fb6ca11cb3" member_type: VOTER last_known_addr { host: "127.19.194.61" port: 39189 } attrs { promote: false } } peers { permanent_uuid: "ad824649dcf140339b33a7ec0f8ebdce" member_type: VOTER last_known_addr { host: "127.19.194.60" port: 41749 } attrs { promote: false } }
I20260806 18:34:40.769743 27404 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "9a38e245bcc34630ba3cc6fb6ca11cb3" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "ad824649dcf140339b33a7ec0f8ebdce"
I20260806 18:34:40.769811 27929 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:34:40.769542 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.194.62:46737 (local address 127.0.0.1:46006)
0806 18:34:40.769695 (+ 153us) negotiation.cc:107] Waiting for socket to connect
0806 18:34:40.769745 (+ 50us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.194.62:46737: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":84}
I20260806 18:34:40.769915 27404 raft_consensus.cc:2379] T 00000000000000000000000000000000 P ad824649dcf140339b33a7ec0f8ebdce [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate 9a38e245bcc34630ba3cc6fb6ca11cb3 in current term 2: Already voted for candidate ad824649dcf140339b33a7ec0f8ebdce in this term.
I20260806 18:34:40.769951 27926 leader_election.cc:288] T 00000000000000000000000000000000 P ad824649dcf140339b33a7ec0f8ebdce [CANDIDATE]: Term 2 election: Requested vote from peers a475b841d65e4ef59cb6d53d3e454091 (127.19.194.62:46737), 9a38e245bcc34630ba3cc6fb6ca11cb3 (127.19.194.61:39189)
I20260806 18:34:40.770133 27217 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "ad824649dcf140339b33a7ec0f8ebdce" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "9a38e245bcc34630ba3cc6fb6ca11cb3"
I20260806 18:34:40.770291 27217 raft_consensus.cc:2379] T 00000000000000000000000000000000 P 9a38e245bcc34630ba3cc6fb6ca11cb3 [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate ad824649dcf140339b33a7ec0f8ebdce in current term 2: Already voted for candidate 9a38e245bcc34630ba3cc6fb6ca11cb3 in this term.
I20260806 18:34:40.770438 27931 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:34:40.770197 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.194.62:46737 (local address 127.0.0.1:46022)
0806 18:34:40.770327 (+ 130us) negotiation.cc:107] Waiting for socket to connect
0806 18:34:40.770380 (+ 53us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.194.62:46737: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":76}
I20260806 18:34:40.770742 27929 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:34:40.770488 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.194.62:46737 (local address 127.0.0.1:46026)
0806 18:34:40.770646 (+ 158us) negotiation.cc:107] Waiting for socket to connect
0806 18:34:40.770690 (+ 44us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.194.62:46737: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":99}
W20260806 18:34:40.770848 27194 leader_election.cc:334] T 00000000000000000000000000000000 P 9a38e245bcc34630ba3cc6fb6ca11cb3 [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer a475b841d65e4ef59cb6d53d3e454091 (127.19.194.62:46737): Network error: Client connection negotiation failed: client connection to 127.19.194.62:46737: connect: Connection refused (error 111)
I20260806 18:34:40.770946 27194 leader_election.cc:302] T 00000000000000000000000000000000 P 9a38e245bcc34630ba3cc6fb6ca11cb3 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9a38e245bcc34630ba3cc6fb6ca11cb3; no voters: a475b841d65e4ef59cb6d53d3e454091, ad824649dcf140339b33a7ec0f8ebdce
I20260806 18:34:40.771040 27931 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:34:40.770852 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.194.62:46737 (local address 127.0.0.1:46038)
0806 18:34:40.770935 (+ 83us) negotiation.cc:107] Waiting for socket to connect
0806 18:34:40.770985 (+ 50us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.194.62:46737: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":24}
I20260806 18:34:40.771076 27927 raft_consensus.cc:2726] T 00000000000000000000000000000000 P 9a38e245bcc34630ba3cc6fb6ca11cb3 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
W20260806 18:34:40.771200 27382 leader_election.cc:334] T 00000000000000000000000000000000 P ad824649dcf140339b33a7ec0f8ebdce [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer a475b841d65e4ef59cb6d53d3e454091 (127.19.194.62:46737): Network error: Client connection negotiation failed: client connection to 127.19.194.62:46737: connect: Connection refused (error 111)
I20260806 18:34:40.771272 27382 leader_election.cc:302] T 00000000000000000000000000000000 P ad824649dcf140339b33a7ec0f8ebdce [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ad824649dcf140339b33a7ec0f8ebdce; no voters: 9a38e245bcc34630ba3cc6fb6ca11cb3, a475b841d65e4ef59cb6d53d3e454091
I20260806 18:34:40.771416 27926 raft_consensus.cc:2726] T 00000000000000000000000000000000 P ad824649dcf140339b33a7ec0f8ebdce [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260806 18:34:41.247484 27936 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:34:41.246942 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.194.62:46737 (local address 127.19.194.1:34943)
0806 18:34:41.247278 (+ 336us) negotiation.cc:107] Waiting for socket to connect
0806 18:34:41.247357 (+ 79us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.194.62:46737: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":234,"thread_start_us":110,"threads_started":1}
I20260806 18:34:41.249686 27937 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:34:41.249212 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.194.62:46737 (local address 127.19.194.2:36879)
0806 18:34:41.249510 (+ 298us) negotiation.cc:107] Waiting for socket to connect
0806 18:34:41.249584 (+ 74us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.194.62:46737: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":222,"thread_start_us":74,"threads_started":1}
I20260806 18:34:41.249722 27938 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:34:41.249212 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.194.62:46737 (local address 127.19.194.3:55007)
0806 18:34:41.249550 (+ 338us) negotiation.cc:107] Waiting for socket to connect
0806 18:34:41.249630 (+ 80us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.194.62:46737: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":256,"thread_start_us":71,"threads_started":1}
I20260806 18:34:42.248981 27939 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:34:42.248469 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.194.62:46737 (local address 127.19.194.1:40779)
0806 18:34:42.248794 (+ 325us) negotiation.cc:107] Waiting for socket to connect
0806 18:34:42.248865 (+ 71us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.194.62:46737: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":240,"thread_start_us":102,"threads_started":1}
I20260806 18:34:42.250773 27940 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:34:42.250282 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.194.62:46737 (local address 127.19.194.3:57913)
0806 18:34:42.250618 (+ 336us) negotiation.cc:107] Waiting for socket to connect
0806 18:34:42.250677 (+ 59us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.194.62:46737: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":261,"thread_start_us":62,"threads_started":1}
I20260806 18:34:42.250825 27941 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:34:42.250376 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.194.62:46737 (local address 127.19.194.2:53423)
0806 18:34:42.250658 (+ 282us) negotiation.cc:107] Waiting for socket to connect
0806 18:34:42.250721 (+ 63us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.194.62:46737: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":202,"thread_start_us":64,"threads_started":1}
I20260806 18:34:42.413491 27942 raft_consensus.cc:482] T 00000000000000000000000000000000 P 9a38e245bcc34630ba3cc6fb6ca11cb3 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 18:34:42.413630 27942 raft_consensus.cc:504] T 00000000000000000000000000000000 P 9a38e245bcc34630ba3cc6fb6ca11cb3 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "a475b841d65e4ef59cb6d53d3e454091" member_type: VOTER last_known_addr { host: "127.19.194.62" port: 46737 } } peers { permanent_uuid: "9a38e245bcc34630ba3cc6fb6ca11cb3" member_type: VOTER last_known_addr { host: "127.19.194.61" port: 39189 } attrs { promote: false } } peers { permanent_uuid: "ad824649dcf140339b33a7ec0f8ebdce" member_type: VOTER last_known_addr { host: "127.19.194.60" port: 41749 } attrs { promote: false } }
I20260806 18:34:42.414067 27942 leader_election.cc:288] T 00000000000000000000000000000000 P 9a38e245bcc34630ba3cc6fb6ca11cb3 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers a475b841d65e4ef59cb6d53d3e454091 (127.19.194.62:46737), ad824649dcf140339b33a7ec0f8ebdce (127.19.194.60:41749)
I20260806 18:34:42.414608 27404 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "9a38e245bcc34630ba3cc6fb6ca11cb3" candidate_term: 3 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "ad824649dcf140339b33a7ec0f8ebdce" is_pre_election: true
I20260806 18:34:42.414693 27943 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:34:42.414246 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.194.62:46737 (local address 127.0.0.1:46282)
0806 18:34:42.414543 (+ 297us) negotiation.cc:107] Waiting for socket to connect
0806 18:34:42.414604 (+ 61us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.194.62:46737: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":214,"thread_start_us":61,"threads_started":1}
I20260806 18:34:42.414767 27404 raft_consensus.cc:2454] T 00000000000000000000000000000000 P ad824649dcf140339b33a7ec0f8ebdce [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9a38e245bcc34630ba3cc6fb6ca11cb3 in term 2.
I20260806 18:34:42.415185 27191 leader_election.cc:302] T 00000000000000000000000000000000 P 9a38e245bcc34630ba3cc6fb6ca11cb3 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9a38e245bcc34630ba3cc6fb6ca11cb3, ad824649dcf140339b33a7ec0f8ebdce; no voters:
I20260806 18:34:42.415396 27942 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 9a38e245bcc34630ba3cc6fb6ca11cb3 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260806 18:34:42.415479 27942 raft_consensus.cc:482] T 00000000000000000000000000000000 P 9a38e245bcc34630ba3cc6fb6ca11cb3 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 18:34:42.415540 27942 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 9a38e245bcc34630ba3cc6fb6ca11cb3 [term 2 FOLLOWER]: Advancing to term 3
I20260806 18:34:42.415693 27943 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:34:42.415408 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.194.62:46737 (local address 127.0.0.1:46286)
0806 18:34:42.415565 (+ 157us) negotiation.cc:107] Waiting for socket to connect
0806 18:34:42.415624 (+ 59us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.194.62:46737: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":70}
W20260806 18:34:42.415804 27194 leader_election.cc:334] T 00000000000000000000000000000000 P 9a38e245bcc34630ba3cc6fb6ca11cb3 [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer a475b841d65e4ef59cb6d53d3e454091 (127.19.194.62:46737): Network error: Client connection negotiation failed: client connection to 127.19.194.62:46737: connect: Connection refused (error 111)
I20260806 18:34:42.416692 27942 raft_consensus.cc:504] T 00000000000000000000000000000000 P 9a38e245bcc34630ba3cc6fb6ca11cb3 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "a475b841d65e4ef59cb6d53d3e454091" member_type: VOTER last_known_addr { host: "127.19.194.62" port: 46737 } } peers { permanent_uuid: "9a38e245bcc34630ba3cc6fb6ca11cb3" member_type: VOTER last_known_addr { host: "127.19.194.61" port: 39189 } attrs { promote: false } } peers { permanent_uuid: "ad824649dcf140339b33a7ec0f8ebdce" member_type: VOTER last_known_addr { host: "127.19.194.60" port: 41749 } attrs { promote: false } }
I20260806 18:34:42.417049 27942 leader_election.cc:288] T 00000000000000000000000000000000 P 9a38e245bcc34630ba3cc6fb6ca11cb3 [CANDIDATE]: Term 3 election: Requested vote from peers a475b841d65e4ef59cb6d53d3e454091 (127.19.194.62:46737), ad824649dcf140339b33a7ec0f8ebdce (127.19.194.60:41749)
I20260806 18:34:42.417399 27943 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:34:42.417188 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.194.62:46737 (local address 127.0.0.1:46294)
0806 18:34:42.417276 (+ 88us) negotiation.cc:107] Waiting for socket to connect
0806 18:34:42.417340 (+ 64us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.194.62:46737: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":25}
I20260806 18:34:42.417426 27404 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "9a38e245bcc34630ba3cc6fb6ca11cb3" candidate_term: 3 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "ad824649dcf140339b33a7ec0f8ebdce"
I20260806 18:34:42.417590 27404 raft_consensus.cc:3037] T 00000000000000000000000000000000 P ad824649dcf140339b33a7ec0f8ebdce [term 2 FOLLOWER]: Advancing to term 3
I20260806 18:34:42.418186 27943 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:34:42.417936 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.194.62:46737 (local address 127.0.0.1:46304)
0806 18:34:42.418095 (+ 159us) negotiation.cc:107] Waiting for socket to connect
0806 18:34:42.418138 (+ 43us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.194.62:46737: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":101}
W20260806 18:34:42.418345 27194 leader_election.cc:334] T 00000000000000000000000000000000 P 9a38e245bcc34630ba3cc6fb6ca11cb3 [CANDIDATE]: Term 3 election: RPC error from VoteRequest() call to peer a475b841d65e4ef59cb6d53d3e454091 (127.19.194.62:46737): Network error: Client connection negotiation failed: client connection to 127.19.194.62:46737: connect: Connection refused (error 111)
I20260806 18:34:42.418782 27404 raft_consensus.cc:2454] T 00000000000000000000000000000000 P ad824649dcf140339b33a7ec0f8ebdce [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9a38e245bcc34630ba3cc6fb6ca11cb3 in term 3.
I20260806 18:34:42.419070 27191 leader_election.cc:302] T 00000000000000000000000000000000 P 9a38e245bcc34630ba3cc6fb6ca11cb3 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9a38e245bcc34630ba3cc6fb6ca11cb3, ad824649dcf140339b33a7ec0f8ebdce; no voters: a475b841d65e4ef59cb6d53d3e454091
I20260806 18:34:42.419269 27942 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 9a38e245bcc34630ba3cc6fb6ca11cb3 [term 3 FOLLOWER]: Leader election won for term 3
I20260806 18:34:42.419440 27942 raft_consensus.cc:686] T 00000000000000000000000000000000 P 9a38e245bcc34630ba3cc6fb6ca11cb3 [term 3 LEADER]: Becoming Leader. State: Replica: 9a38e245bcc34630ba3cc6fb6ca11cb3, State: Running, Role: LEADER
I20260806 18:34:42.419711 27942 consensus_queue.cc:237] T 00000000000000000000000000000000 P 9a38e245bcc34630ba3cc6fb6ca11cb3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 1.12, Last appended by leader: 12, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "a475b841d65e4ef59cb6d53d3e454091" member_type: VOTER last_known_addr { host: "127.19.194.62" port: 46737 } } peers { permanent_uuid: "9a38e245bcc34630ba3cc6fb6ca11cb3" member_type: VOTER last_known_addr { host: "127.19.194.61" port: 39189 } attrs { promote: false } } peers { permanent_uuid: "ad824649dcf140339b33a7ec0f8ebdce" member_type: VOTER last_known_addr { host: "127.19.194.60" port: 41749 } attrs { promote: false } }
I20260806 18:34:42.420706 27945 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9a38e245bcc34630ba3cc6fb6ca11cb3 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 9a38e245bcc34630ba3cc6fb6ca11cb3. Latest consensus state: current_term: 3 leader_uuid: "9a38e245bcc34630ba3cc6fb6ca11cb3" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "a475b841d65e4ef59cb6d53d3e454091" member_type: VOTER last_known_addr { host: "127.19.194.62" port: 46737 } } peers { permanent_uuid: "9a38e245bcc34630ba3cc6fb6ca11cb3" member_type: VOTER last_known_addr { host: "127.19.194.61" port: 39189 } attrs { promote: false } } peers { permanent_uuid: "ad824649dcf140339b33a7ec0f8ebdce" member_type: VOTER last_known_addr { host: "127.19.194.60" port: 41749 } attrs { promote: false } } }
I20260806 18:34:42.420845 27945 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9a38e245bcc34630ba3cc6fb6ca11cb3 [sys.catalog]: This master's current role is: LEADER
I20260806 18:34:42.421201 27947 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 18:34:42.423822 27947 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 18:34:42.424320 27947 catalog_manager.cc:1259] Loaded cluster ID: 3fbbab1f31df424cb2df0f828bded742
I20260806 18:34:42.424373 27947 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 18:34:42.425076 27947 catalog_manager.cc:1504] Loading token signing keys...
I20260806 18:34:42.425501 27947 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 9a38e245bcc34630ba3cc6fb6ca11cb3: Loaded TSK: 0
I20260806 18:34:42.426168 27947 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 18:34:42.506983 27199 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:37874:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260806 18:34:42.510249 27199 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 18:34:42.513450 27943 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:34:42.513189 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.194.62:46737 (local address 127.0.0.1:46326)
0806 18:34:42.513319 (+ 130us) negotiation.cc:107] Waiting for socket to connect
0806 18:34:42.513365 (+ 46us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.194.62:46737: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":51}
I20260806 18:34:42.513512 27404 raft_consensus.cc:1261] T 00000000000000000000000000000000 P ad824649dcf140339b33a7ec0f8ebdce [term 3 FOLLOWER]: Refusing update from remote peer 9a38e245bcc34630ba3cc6fb6ca11cb3: Log matching property violated. Preceding OpId in replica: term: 1 index: 12. Preceding OpId from leader: term: 3 index: 14. (index mismatch)
I20260806 18:34:42.514038 27942 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 9a38e245bcc34630ba3cc6fb6ca11cb3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ad824649dcf140339b33a7ec0f8ebdce" member_type: VOTER last_known_addr { host: "127.19.194.60" port: 41749 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20260806 18:34:42.514200 27943 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:34:42.513936 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.194.62:46737 (local address 127.0.0.1:46334)
0806 18:34:42.514072 (+ 136us) negotiation.cc:107] Waiting for socket to connect
0806 18:34:42.514136 (+ 64us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.194.62:46737: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":52}
W20260806 18:34:42.514355 27194 consensus_peers.cc:487] T 00000000000000000000000000000000 P 9a38e245bcc34630ba3cc6fb6ca11cb3 -> Peer a475b841d65e4ef59cb6d53d3e454091 (127.19.194.62:46737): Couldn't send request to peer a475b841d65e4ef59cb6d53d3e454091. Status: Network error: Client connection negotiation failed: client connection to 127.19.194.62:46737: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260806 18:34:42.515662 27949 sys_catalog.cc:455] T 00000000000000000000000000000000 P ad824649dcf140339b33a7ec0f8ebdce [sys.catalog]: SysCatalogTable state changed. Reason: New leader 9a38e245bcc34630ba3cc6fb6ca11cb3. Latest consensus state: current_term: 3 leader_uuid: "9a38e245bcc34630ba3cc6fb6ca11cb3" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "a475b841d65e4ef59cb6d53d3e454091" member_type: VOTER last_known_addr { host: "127.19.194.62" port: 46737 } } peers { permanent_uuid: "9a38e245bcc34630ba3cc6fb6ca11cb3" member_type: VOTER last_known_addr { host: "127.19.194.61" port: 39189 } attrs { promote: false } } peers { permanent_uuid: "ad824649dcf140339b33a7ec0f8ebdce" member_type: VOTER last_known_addr { host: "127.19.194.60" port: 41749 } attrs { promote: false } } }
I20260806 18:34:42.515807 27949 sys_catalog.cc:458] T 00000000000000000000000000000000 P ad824649dcf140339b33a7ec0f8ebdce [sys.catalog]: This master's current role is: FOLLOWER
I20260806 18:34:42.516937 27945 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9a38e245bcc34630ba3cc6fb6ca11cb3 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 3 leader_uuid: "9a38e245bcc34630ba3cc6fb6ca11cb3" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "a475b841d65e4ef59cb6d53d3e454091" member_type: VOTER last_known_addr { host: "127.19.194.62" port: 46737 } } peers { permanent_uuid: "9a38e245bcc34630ba3cc6fb6ca11cb3" member_type: VOTER last_known_addr { host: "127.19.194.61" port: 39189 } attrs { promote: false } } peers { permanent_uuid: "ad824649dcf140339b33a7ec0f8ebdce" member_type: VOTER last_known_addr { host: "127.19.194.60" port: 41749 } attrs { promote: false } } }
I20260806 18:34:42.517048 27945 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9a38e245bcc34630ba3cc6fb6ca11cb3 [sys.catalog]: This master's current role is: LEADER
I20260806 18:34:42.517624 27949 sys_catalog.cc:455] T 00000000000000000000000000000000 P ad824649dcf140339b33a7ec0f8ebdce [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 3 leader_uuid: "9a38e245bcc34630ba3cc6fb6ca11cb3" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "a475b841d65e4ef59cb6d53d3e454091" member_type: VOTER last_known_addr { host: "127.19.194.62" port: 46737 } } peers { permanent_uuid: "9a38e245bcc34630ba3cc6fb6ca11cb3" member_type: VOTER last_known_addr { host: "127.19.194.61" port: 39189 } attrs { promote: false } } peers { permanent_uuid: "ad824649dcf140339b33a7ec0f8ebdce" member_type: VOTER last_known_addr { host: "127.19.194.60" port: 41749 } attrs { promote: false } } }
I20260806 18:34:42.517751 27949 sys_catalog.cc:458] T 00000000000000000000000000000000 P ad824649dcf140339b33a7ec0f8ebdce [sys.catalog]: This master's current role is: FOLLOWER
I20260806 18:34:42.518110 27950 mvcc.cc:205] Tried to move back new op lower bound from 7315625093169856512 to 7315625092793278464. Current Snapshot: MvccSnapshot[applied={T|T < 7315625093169856512 or (T in {7315625093169856512})}]
I20260806 18:34:42.527820 27943 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:34:42.522844 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.194.2:44715 (local address 127.0.0.1:40534)
0806 18:34:42.522983 (+ 139us) negotiation.cc:107] Waiting for socket to connect
0806 18:34:42.522997 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0806 18:34:42.523097 (+ 100us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:34:42.524351 (+ 1254us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:34:42.524355 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:34:42.524371 (+ 16us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 18:34:42.524465 (+ 94us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:34:42.524471 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:34:42.525379 (+ 908us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:34:42.525382 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:34:42.526457 (+ 1075us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:34:42.526467 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:34:42.527406 (+ 939us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:34:42.527410 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:34:42.527497 (+ 87us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:34:42.527520 (+ 23us) client_negotiation.cc:769] Sending connection context
0806 18:34:42.527635 (+ 115us) client_negotiation.cc:240] Negotiation successful
0806 18:34:42.527705 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":67}
I20260806 18:34:42.527861 27957 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:34:42.523814 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40534 (local address 127.19.194.2:44715)
0806 18:34:42.524071 (+ 257us) server_negotiation.cc:206] Beginning negotiation
0806 18:34:42.524076 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 18:34:42.524090 (+ 14us) server_negotiation.cc:407] Connection header received
0806 18:34:42.524144 (+ 54us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:34:42.524147 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:34:42.524204 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:34:42.524341 (+ 137us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 18:34:42.524589 (+ 248us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:34:42.525262 (+ 673us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:34:42.526608 (+ 1346us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:34:42.527281 (+ 673us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:34:42.527405 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:34:42.527468 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0806 18:34:42.527653 (+ 185us) server_negotiation.cc:299] Negotiation successful
0806 18:34:42.527728 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":184,"thread_start_us":114,"threads_started":1}
I20260806 18:34:42.528402 27956 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:34:42.523473 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.194.3:42189 (local address 127.0.0.1:35488)
0806 18:34:42.523996 (+ 523us) negotiation.cc:107] Waiting for socket to connect
0806 18:34:42.524012 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0806 18:34:42.524095 (+ 83us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:34:42.524577 (+ 482us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:34:42.524580 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:34:42.524597 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 18:34:42.524690 (+ 93us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:34:42.524695 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:34:42.525846 (+ 1151us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:34:42.525849 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:34:42.526960 (+ 1111us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:34:42.526969 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:34:42.527984 (+ 1015us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:34:42.527988 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:34:42.528083 (+ 95us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:34:42.528114 (+ 31us) client_negotiation.cc:769] Sending connection context
0806 18:34:42.528199 (+ 85us) client_negotiation.cc:240] Negotiation successful
0806 18:34:42.528272 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":449,"thread_start_us":105,"threads_started":1}
I20260806 18:34:42.528507 27955 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:34:42.523460 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.194.1:33871 (local address 127.0.0.1:35148)
0806 18:34:42.523714 (+ 254us) negotiation.cc:107] Waiting for socket to connect
0806 18:34:42.523729 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0806 18:34:42.523835 (+ 106us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:34:42.524871 (+ 1036us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:34:42.524874 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:34:42.524913 (+ 39us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 18:34:42.525009 (+ 96us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:34:42.525015 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:34:42.526154 (+ 1139us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:34:42.526158 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:34:42.527115 (+ 957us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:34:42.527126 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:34:42.528044 (+ 918us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:34:42.528049 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:34:42.528182 (+ 133us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:34:42.528205 (+ 23us) client_negotiation.cc:769] Sending connection context
0806 18:34:42.528254 (+ 49us) client_negotiation.cc:240] Negotiation successful
0806 18:34:42.528335 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":174,"thread_start_us":98,"threads_started":1}
I20260806 18:34:42.528524 27959 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:34:42.524096 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35488 (local address 127.19.194.3:42189)
0806 18:34:42.524355 (+ 259us) server_negotiation.cc:206] Beginning negotiation
0806 18:34:42.524360 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 18:34:42.524373 (+ 13us) server_negotiation.cc:407] Connection header received
0806 18:34:42.524423 (+ 50us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:34:42.524426 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:34:42.524473 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:34:42.524563 (+ 90us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 18:34:42.524818 (+ 255us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:34:42.525721 (+ 903us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:34:42.527090 (+ 1369us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:34:42.527731 (+ 641us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:34:42.527845 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:34:42.528026 (+ 181us) server_negotiation.cc:1035] Waiting for connection context
0806 18:34:42.528319 (+ 293us) server_negotiation.cc:299] Negotiation successful
0806 18:34:42.528395 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":180,"thread_start_us":96,"threads_started":1}
I20260806 18:34:42.528662 27651 tablet_service.cc:1467] Processing CreateTablet for tablet 06184b09bc26427e8636c94069d9c309 (DEFAULT_TABLE table=test-table [id=c8d1a5008685469b85bf57eff0b0b0a7]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 18:34:42.528945 27958 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:34:42.523902 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35148 (local address 127.19.194.1:33871)
0806 18:34:42.524347 (+ 445us) server_negotiation.cc:206] Beginning negotiation
0806 18:34:42.524353 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 18:34:42.524368 (+ 15us) server_negotiation.cc:407] Connection header received
0806 18:34:42.524423 (+ 55us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:34:42.524426 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:34:42.524473 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:34:42.525126 (+ 653us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 18:34:42.525238 (+ 112us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:34:42.526016 (+ 778us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:34:42.527297 (+ 1281us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:34:42.527904 (+ 607us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:34:42.528684 (+ 780us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:34:42.528763 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0806 18:34:42.528796 (+ 33us) server_negotiation.cc:299] Negotiation successful
0806 18:34:42.528850 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":358,"thread_start_us":76,"threads_started":1}
I20260806 18:34:42.528980 27651 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 06184b09bc26427e8636c94069d9c309. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 18:34:42.529487 27511 tablet_service.cc:1467] Processing CreateTablet for tablet 06184b09bc26427e8636c94069d9c309 (DEFAULT_TABLE table=test-table [id=c8d1a5008685469b85bf57eff0b0b0a7]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 18:34:42.529758 27511 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 06184b09bc26427e8636c94069d9c309. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 18:34:42.531040 27960 tablet_bootstrap.cc:492] T 06184b09bc26427e8636c94069d9c309 P a5630d59c6324f3b87d9d2a1f4024c45: Bootstrap starting.
I20260806 18:34:42.532097 27960 tablet_bootstrap.cc:654] T 06184b09bc26427e8636c94069d9c309 P a5630d59c6324f3b87d9d2a1f4024c45: Neither blocks nor log segments found. Creating new log.
I20260806 18:34:42.532598 27789 tablet_service.cc:1467] Processing CreateTablet for tablet 06184b09bc26427e8636c94069d9c309 (DEFAULT_TABLE table=test-table [id=c8d1a5008685469b85bf57eff0b0b0a7]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 18:34:42.532963 27789 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 06184b09bc26427e8636c94069d9c309. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 18:34:42.533548 27960 tablet_bootstrap.cc:492] T 06184b09bc26427e8636c94069d9c309 P a5630d59c6324f3b87d9d2a1f4024c45: No bootstrap required, opened a new log
I20260806 18:34:42.533666 27960 ts_tablet_manager.cc:1397] T 06184b09bc26427e8636c94069d9c309 P a5630d59c6324f3b87d9d2a1f4024c45: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260806 18:34:42.534103 27960 raft_consensus.cc:348] T 06184b09bc26427e8636c94069d9c309 P a5630d59c6324f3b87d9d2a1f4024c45 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a5630d59c6324f3b87d9d2a1f4024c45" member_type: VOTER last_known_addr { host: "127.19.194.2" port: 44715 } } peers { permanent_uuid: "402d0dd8fe1f4b4385b0b94ba213aaa6" member_type: VOTER last_known_addr { host: "127.19.194.3" port: 42189 } } peers { permanent_uuid: "a7dbdd68b24540cda5ada7e1e0d73ccd" member_type: VOTER last_known_addr { host: "127.19.194.1" port: 33871 } }
I20260806 18:34:42.534209 27960 raft_consensus.cc:374] T 06184b09bc26427e8636c94069d9c309 P a5630d59c6324f3b87d9d2a1f4024c45 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 18:34:42.534232 27960 raft_consensus.cc:729] T 06184b09bc26427e8636c94069d9c309 P a5630d59c6324f3b87d9d2a1f4024c45 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a5630d59c6324f3b87d9d2a1f4024c45, State: Initialized, Role: FOLLOWER
I20260806 18:34:42.534369 27960 consensus_queue.cc:260] T 06184b09bc26427e8636c94069d9c309 P a5630d59c6324f3b87d9d2a1f4024c45 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a5630d59c6324f3b87d9d2a1f4024c45" member_type: VOTER last_known_addr { host: "127.19.194.2" port: 44715 } } peers { permanent_uuid: "402d0dd8fe1f4b4385b0b94ba213aaa6" member_type: VOTER last_known_addr { host: "127.19.194.3" port: 42189 } } peers { permanent_uuid: "a7dbdd68b24540cda5ada7e1e0d73ccd" member_type: VOTER last_known_addr { host: "127.19.194.1" port: 33871 } }
I20260806 18:34:42.534715 27960 ts_tablet_manager.cc:1428] T 06184b09bc26427e8636c94069d9c309 P a5630d59c6324f3b87d9d2a1f4024c45: Time spent starting tablet: real 0.001s user 0.002s sys 0.000s
I20260806 18:34:42.535372 27941 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:34:42.535163 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.194.62:46737 (local address 127.19.194.2:39401)
0806 18:34:42.535260 (+ 97us) negotiation.cc:107] Waiting for socket to connect
0806 18:34:42.535308 (+ 48us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.194.62:46737: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":26}
I20260806 18:34:42.535573 27964 tablet_bootstrap.cc:492] T 06184b09bc26427e8636c94069d9c309 P 402d0dd8fe1f4b4385b0b94ba213aaa6: Bootstrap starting.
I20260806 18:34:42.536111 27961 tablet_bootstrap.cc:492] T 06184b09bc26427e8636c94069d9c309 P a7dbdd68b24540cda5ada7e1e0d73ccd: Bootstrap starting.
I20260806 18:34:42.536491 27964 tablet_bootstrap.cc:654] T 06184b09bc26427e8636c94069d9c309 P 402d0dd8fe1f4b4385b0b94ba213aaa6: Neither blocks nor log segments found. Creating new log.
I20260806 18:34:42.537022 27961 tablet_bootstrap.cc:654] T 06184b09bc26427e8636c94069d9c309 P a7dbdd68b24540cda5ada7e1e0d73ccd: Neither blocks nor log segments found. Creating new log.
I20260806 18:34:42.537770 27964 tablet_bootstrap.cc:492] T 06184b09bc26427e8636c94069d9c309 P 402d0dd8fe1f4b4385b0b94ba213aaa6: No bootstrap required, opened a new log
I20260806 18:34:42.537914 27964 ts_tablet_manager.cc:1397] T 06184b09bc26427e8636c94069d9c309 P 402d0dd8fe1f4b4385b0b94ba213aaa6: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260806 18:34:42.538338 27961 tablet_bootstrap.cc:492] T 06184b09bc26427e8636c94069d9c309 P a7dbdd68b24540cda5ada7e1e0d73ccd: No bootstrap required, opened a new log
I20260806 18:34:42.538436 27961 ts_tablet_manager.cc:1397] T 06184b09bc26427e8636c94069d9c309 P a7dbdd68b24540cda5ada7e1e0d73ccd: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260806 18:34:42.538435 27964 raft_consensus.cc:348] T 06184b09bc26427e8636c94069d9c309 P 402d0dd8fe1f4b4385b0b94ba213aaa6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a5630d59c6324f3b87d9d2a1f4024c45" member_type: VOTER last_known_addr { host: "127.19.194.2" port: 44715 } } peers { permanent_uuid: "402d0dd8fe1f4b4385b0b94ba213aaa6" member_type: VOTER last_known_addr { host: "127.19.194.3" port: 42189 } } peers { permanent_uuid: "a7dbdd68b24540cda5ada7e1e0d73ccd" member_type: VOTER last_known_addr { host: "127.19.194.1" port: 33871 } }
I20260806 18:34:42.538643 27964 raft_consensus.cc:374] T 06184b09bc26427e8636c94069d9c309 P 402d0dd8fe1f4b4385b0b94ba213aaa6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 18:34:42.538710 27964 raft_consensus.cc:729] T 06184b09bc26427e8636c94069d9c309 P 402d0dd8fe1f4b4385b0b94ba213aaa6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 402d0dd8fe1f4b4385b0b94ba213aaa6, State: Initialized, Role: FOLLOWER
I20260806 18:34:42.538854 27941 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:34:42.538383 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.194.62:46737 (local address 127.19.194.2:46673)
0806 18:34:42.538731 (+ 348us) negotiation.cc:107] Waiting for socket to connect
0806 18:34:42.538789 (+ 58us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.194.62:46737: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":265}
I20260806 18:34:42.538880 27964 consensus_queue.cc:260] T 06184b09bc26427e8636c94069d9c309 P 402d0dd8fe1f4b4385b0b94ba213aaa6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a5630d59c6324f3b87d9d2a1f4024c45" member_type: VOTER last_known_addr { host: "127.19.194.2" port: 44715 } } peers { permanent_uuid: "402d0dd8fe1f4b4385b0b94ba213aaa6" member_type: VOTER last_known_addr { host: "127.19.194.3" port: 42189 } } peers { permanent_uuid: "a7dbdd68b24540cda5ada7e1e0d73ccd" member_type: VOTER last_known_addr { host: "127.19.194.1" port: 33871 } }
I20260806 18:34:42.538892 27961 raft_consensus.cc:348] T 06184b09bc26427e8636c94069d9c309 P a7dbdd68b24540cda5ada7e1e0d73ccd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a5630d59c6324f3b87d9d2a1f4024c45" member_type: VOTER last_known_addr { host: "127.19.194.2" port: 44715 } } peers { permanent_uuid: "402d0dd8fe1f4b4385b0b94ba213aaa6" member_type: VOTER last_known_addr { host: "127.19.194.3" port: 42189 } } peers { permanent_uuid: "a7dbdd68b24540cda5ada7e1e0d73ccd" member_type: VOTER last_known_addr { host: "127.19.194.1" port: 33871 } }
I20260806 18:34:42.539032 27961 raft_consensus.cc:374] T 06184b09bc26427e8636c94069d9c309 P a7dbdd68b24540cda5ada7e1e0d73ccd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 18:34:42.539093 27961 raft_consensus.cc:729] T 06184b09bc26427e8636c94069d9c309 P a7dbdd68b24540cda5ada7e1e0d73ccd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a7dbdd68b24540cda5ada7e1e0d73ccd, State: Initialized, Role: FOLLOWER
I20260806 18:34:42.539258 27961 consensus_queue.cc:260] T 06184b09bc26427e8636c94069d9c309 P a7dbdd68b24540cda5ada7e1e0d73ccd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a5630d59c6324f3b87d9d2a1f4024c45" member_type: VOTER last_known_addr { host: "127.19.194.2" port: 44715 } } peers { permanent_uuid: "402d0dd8fe1f4b4385b0b94ba213aaa6" member_type: VOTER last_known_addr { host: "127.19.194.3" port: 42189 } } peers { permanent_uuid: "a7dbdd68b24540cda5ada7e1e0d73ccd" member_type: VOTER last_known_addr { host: "127.19.194.1" port: 33871 } }
I20260806 18:34:42.539415 27964 ts_tablet_manager.cc:1428] T 06184b09bc26427e8636c94069d9c309 P 402d0dd8fe1f4b4385b0b94ba213aaa6: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260806 18:34:42.539636 27961 ts_tablet_manager.cc:1428] T 06184b09bc26427e8636c94069d9c309 P a7dbdd68b24540cda5ada7e1e0d73ccd: Time spent starting tablet: real 0.001s user 0.002s sys 0.000s
I20260806 18:34:42.540323 27940 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:34:42.540099 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.194.62:46737 (local address 127.19.194.3:50907)
0806 18:34:42.540202 (+ 103us) negotiation.cc:107] Waiting for socket to connect
0806 18:34:42.540257 (+ 55us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.194.62:46737: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":31}
I20260806 18:34:42.540977 27939 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:34:42.540697 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.194.62:46737 (local address 127.19.194.1:51989)
0806 18:34:42.540846 (+ 149us) negotiation.cc:107] Waiting for socket to connect
0806 18:34:42.540900 (+ 54us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.194.62:46737: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":84}
I20260806 18:34:42.541335 27940 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:34:42.541058 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.194.62:46737 (local address 127.19.194.3:48729)
0806 18:34:42.541219 (+ 161us) negotiation.cc:107] Waiting for socket to connect
0806 18:34:42.541267 (+ 48us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.194.62:46737: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":93}
I20260806 18:34:42.542115 27939 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:34:42.541865 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.194.62:46737 (local address 127.19.194.1:43031)
0806 18:34:42.542003 (+ 138us) negotiation.cc:107] Waiting for socket to connect
0806 18:34:42.542055 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.194.62:46737: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":76}
I20260806 18:34:42.542850 27580 heartbeater.cc:503] Master 127.19.194.61:39189 was elected leader, sending a full tablet report...
I20260806 18:34:42.552728 27963 raft_consensus.cc:482] T 06184b09bc26427e8636c94069d9c309 P a5630d59c6324f3b87d9d2a1f4024c45 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 18:34:42.552839 27963 raft_consensus.cc:504] T 06184b09bc26427e8636c94069d9c309 P a5630d59c6324f3b87d9d2a1f4024c45 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a5630d59c6324f3b87d9d2a1f4024c45" member_type: VOTER last_known_addr { host: "127.19.194.2" port: 44715 } } peers { permanent_uuid: "402d0dd8fe1f4b4385b0b94ba213aaa6" member_type: VOTER last_known_addr { host: "127.19.194.3" port: 42189 } } peers { permanent_uuid: "a7dbdd68b24540cda5ada7e1e0d73ccd" member_type: VOTER last_known_addr { host: "127.19.194.1" port: 33871 } }
I20260806 18:34:42.553846 27963 leader_election.cc:288] T 06184b09bc26427e8636c94069d9c309 P a5630d59c6324f3b87d9d2a1f4024c45 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 402d0dd8fe1f4b4385b0b94ba213aaa6 (127.19.194.3:42189), a7dbdd68b24540cda5ada7e1e0d73ccd (127.19.194.1:33871)
I20260806 18:34:42.558512 27958 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:34:42.554128 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.194.2:33345 (local address 127.19.194.1:33871)
0806 18:34:42.554239 (+ 111us) server_negotiation.cc:206] Beginning negotiation
0806 18:34:42.554256 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0806 18:34:42.554269 (+ 13us) server_negotiation.cc:407] Connection header received
0806 18:34:42.554364 (+ 95us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:34:42.554367 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:34:42.554414 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:34:42.554486 (+ 72us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 18:34:42.554828 (+ 342us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:34:42.555624 (+ 796us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:34:42.557279 (+ 1655us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:34:42.557881 (+ 602us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:34:42.557999 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:34:42.558046 (+ 47us) server_negotiation.cc:1035] Waiting for connection context
0806 18:34:42.558337 (+ 291us) server_negotiation.cc:299] Negotiation successful
0806 18:34:42.558404 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34}
I20260806 18:34:42.558511 27959 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:34:42.554110 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.194.2:36565 (local address 127.19.194.3:42189)
0806 18:34:42.554269 (+ 159us) server_negotiation.cc:206] Beginning negotiation
0806 18:34:42.554275 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 18:34:42.554533 (+ 258us) server_negotiation.cc:407] Connection header received
0806 18:34:42.554622 (+ 89us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:34:42.554628 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:34:42.554662 (+ 34us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:34:42.554716 (+ 54us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 18:34:42.555008 (+ 292us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:34:42.555748 (+ 740us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:34:42.557279 (+ 1531us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:34:42.557881 (+ 602us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:34:42.558006 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:34:42.558046 (+ 40us) server_negotiation.cc:1035] Waiting for connection context
0806 18:34:42.558337 (+ 291us) server_negotiation.cc:299] Negotiation successful
0806 18:34:42.558404 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":74}
I20260806 18:34:42.558573 27941 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:34:42.554037 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.194.1:33871 (local address 127.19.194.2:33345)
0806 18:34:42.554157 (+ 120us) negotiation.cc:107] Waiting for socket to connect
0806 18:34:42.554173 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0806 18:34:42.554264 (+ 91us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:34:42.554512 (+ 248us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:34:42.554515 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:34:42.554532 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 18:34:42.554677 (+ 145us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:34:42.554682 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:34:42.555722 (+ 1040us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:34:42.555725 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:34:42.557150 (+ 1425us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:34:42.557162 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:34:42.558033 (+ 871us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:34:42.558035 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:34:42.558175 (+ 140us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:34:42.558228 (+ 53us) client_negotiation.cc:769] Sending connection context
0806 18:34:42.558308 (+ 80us) client_negotiation.cc:240] Negotiation successful
0806 18:34:42.558418 (+ 110us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":44,"mutex_wait_us":159}
I20260806 18:34:42.558584 27969 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:34:42.554037 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.194.3:42189 (local address 127.19.194.2:36565)
0806 18:34:42.554423 (+ 386us) negotiation.cc:107] Waiting for socket to connect
0806 18:34:42.554433 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0806 18:34:42.554548 (+ 115us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:34:42.554779 (+ 231us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:34:42.554784 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:34:42.554804 (+ 20us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 18:34:42.554900 (+ 96us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:34:42.554904 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:34:42.555849 (+ 945us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:34:42.555852 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:34:42.557156 (+ 1304us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:34:42.557162 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:34:42.558030 (+ 868us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:34:42.558034 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:34:42.558203 (+ 169us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:34:42.558242 (+ 39us) client_negotiation.cc:769] Sending connection context
0806 18:34:42.558308 (+ 66us) client_negotiation.cc:240] Negotiation successful
0806 18:34:42.558423 (+ 115us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":329,"mutex_wait_us":217,"spinlock_wait_cycles":18816,"thread_start_us":99,"threads_started":1}
I20260806 18:34:42.559115 27531 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "06184b09bc26427e8636c94069d9c309" candidate_uuid: "a5630d59c6324f3b87d9d2a1f4024c45" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a7dbdd68b24540cda5ada7e1e0d73ccd" is_pre_election: true
I20260806 18:34:42.559269 27531 raft_consensus.cc:2454] T 06184b09bc26427e8636c94069d9c309 P a7dbdd68b24540cda5ada7e1e0d73ccd [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a5630d59c6324f3b87d9d2a1f4024c45 in term 0.
I20260806 18:34:42.559408 27809 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "06184b09bc26427e8636c94069d9c309" candidate_uuid: "a5630d59c6324f3b87d9d2a1f4024c45" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "402d0dd8fe1f4b4385b0b94ba213aaa6" is_pre_election: true
I20260806 18:34:42.559625 27809 raft_consensus.cc:2454] T 06184b09bc26427e8636c94069d9c309 P 402d0dd8fe1f4b4385b0b94ba213aaa6 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a5630d59c6324f3b87d9d2a1f4024c45 in term 0.
I20260806 18:34:42.559623 27603 leader_election.cc:302] T 06184b09bc26427e8636c94069d9c309 P a5630d59c6324f3b87d9d2a1f4024c45 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a5630d59c6324f3b87d9d2a1f4024c45, a7dbdd68b24540cda5ada7e1e0d73ccd; no voters:
I20260806 18:34:42.559867 27963 raft_consensus.cc:2781] T 06184b09bc26427e8636c94069d9c309 P a5630d59c6324f3b87d9d2a1f4024c45 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 18:34:42.559929 27963 raft_consensus.cc:482] T 06184b09bc26427e8636c94069d9c309 P a5630d59c6324f3b87d9d2a1f4024c45 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 18:34:42.559983 27963 raft_consensus.cc:3037] T 06184b09bc26427e8636c94069d9c309 P a5630d59c6324f3b87d9d2a1f4024c45 [term 0 FOLLOWER]: Advancing to term 1
I20260806 18:34:42.561000 27963 raft_consensus.cc:504] T 06184b09bc26427e8636c94069d9c309 P a5630d59c6324f3b87d9d2a1f4024c45 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a5630d59c6324f3b87d9d2a1f4024c45" member_type: VOTER last_known_addr { host: "127.19.194.2" port: 44715 } } peers { permanent_uuid: "402d0dd8fe1f4b4385b0b94ba213aaa6" member_type: VOTER last_known_addr { host: "127.19.194.3" port: 42189 } } peers { permanent_uuid: "a7dbdd68b24540cda5ada7e1e0d73ccd" member_type: VOTER last_known_addr { host: "127.19.194.1" port: 33871 } }
I20260806 18:34:42.561344 27963 leader_election.cc:288] T 06184b09bc26427e8636c94069d9c309 P a5630d59c6324f3b87d9d2a1f4024c45 [CANDIDATE]: Term 1 election: Requested vote from peers 402d0dd8fe1f4b4385b0b94ba213aaa6 (127.19.194.3:42189), a7dbdd68b24540cda5ada7e1e0d73ccd (127.19.194.1:33871)
I20260806 18:34:42.561631 27809 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "06184b09bc26427e8636c94069d9c309" candidate_uuid: "a5630d59c6324f3b87d9d2a1f4024c45" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "402d0dd8fe1f4b4385b0b94ba213aaa6"
I20260806 18:34:42.561749 27809 raft_consensus.cc:3037] T 06184b09bc26427e8636c94069d9c309 P 402d0dd8fe1f4b4385b0b94ba213aaa6 [term 0 FOLLOWER]: Advancing to term 1
I20260806 18:34:42.561784 27531 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "06184b09bc26427e8636c94069d9c309" candidate_uuid: "a5630d59c6324f3b87d9d2a1f4024c45" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a7dbdd68b24540cda5ada7e1e0d73ccd"
I20260806 18:34:42.561925 27531 raft_consensus.cc:3037] T 06184b09bc26427e8636c94069d9c309 P a7dbdd68b24540cda5ada7e1e0d73ccd [term 0 FOLLOWER]: Advancing to term 1
I20260806 18:34:42.562604 27809 raft_consensus.cc:2454] T 06184b09bc26427e8636c94069d9c309 P 402d0dd8fe1f4b4385b0b94ba213aaa6 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a5630d59c6324f3b87d9d2a1f4024c45 in term 1.
I20260806 18:34:42.562896 27531 raft_consensus.cc:2454] T 06184b09bc26427e8636c94069d9c309 P a7dbdd68b24540cda5ada7e1e0d73ccd [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a5630d59c6324f3b87d9d2a1f4024c45 in term 1.
I20260806 18:34:42.562906 27605 leader_election.cc:302] T 06184b09bc26427e8636c94069d9c309 P a5630d59c6324f3b87d9d2a1f4024c45 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 402d0dd8fe1f4b4385b0b94ba213aaa6, a5630d59c6324f3b87d9d2a1f4024c45; no voters:
I20260806 18:34:42.563081 27963 raft_consensus.cc:2781] T 06184b09bc26427e8636c94069d9c309 P a5630d59c6324f3b87d9d2a1f4024c45 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 18:34:42.563301 27963 raft_consensus.cc:686] T 06184b09bc26427e8636c94069d9c309 P a5630d59c6324f3b87d9d2a1f4024c45 [term 1 LEADER]: Becoming Leader. State: Replica: a5630d59c6324f3b87d9d2a1f4024c45, State: Running, Role: LEADER
I20260806 18:34:42.563558 27963 consensus_queue.cc:237] T 06184b09bc26427e8636c94069d9c309 P a5630d59c6324f3b87d9d2a1f4024c45 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a5630d59c6324f3b87d9d2a1f4024c45" member_type: VOTER last_known_addr { host: "127.19.194.2" port: 44715 } } peers { permanent_uuid: "402d0dd8fe1f4b4385b0b94ba213aaa6" member_type: VOTER last_known_addr { host: "127.19.194.3" port: 42189 } } peers { permanent_uuid: "a7dbdd68b24540cda5ada7e1e0d73ccd" member_type: VOTER last_known_addr { host: "127.19.194.1" port: 33871 } }
I20260806 18:34:42.563653 27719 heartbeater.cc:503] Master 127.19.194.61:39189 was elected leader, sending a full tablet report...
I20260806 18:34:42.564982 27969 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:34:42.564738 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.194.62:46737 (local address 127.19.194.2:50367)
0806 18:34:42.564862 (+ 124us) negotiation.cc:107] Waiting for socket to connect
0806 18:34:42.564918 (+ 56us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.194.62:46737: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":55}
I20260806 18:34:42.565727 27199 catalog_manager.cc:5495] T 06184b09bc26427e8636c94069d9c309 P a5630d59c6324f3b87d9d2a1f4024c45 reported cstate change: term changed from 0 to 1, leader changed from <none> to a5630d59c6324f3b87d9d2a1f4024c45 (127.19.194.2). New cstate: current_term: 1 leader_uuid: "a5630d59c6324f3b87d9d2a1f4024c45" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a5630d59c6324f3b87d9d2a1f4024c45" member_type: VOTER last_known_addr { host: "127.19.194.2" port: 44715 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "402d0dd8fe1f4b4385b0b94ba213aaa6" member_type: VOTER last_known_addr { host: "127.19.194.3" port: 42189 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a7dbdd68b24540cda5ada7e1e0d73ccd" member_type: VOTER last_known_addr { host: "127.19.194.1" port: 33871 } health_report { overall_health: UNKNOWN } } }
I20260806 18:34:42.577651 27957 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:34:42.574247 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40546 (local address 127.19.194.2:44715)
0806 18:34:42.574594 (+ 347us) server_negotiation.cc:206] Beginning negotiation
0806 18:34:42.574597 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 18:34:42.574607 (+ 10us) server_negotiation.cc:407] Connection header received
0806 18:34:42.574663 (+ 56us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:34:42.574665 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:34:42.574708 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:34:42.574785 (+ 77us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 18:34:42.575052 (+ 267us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:34:42.575609 (+ 557us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:34:42.576452 (+ 843us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:34:42.576657 (+ 205us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:34:42.576731 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:34:42.576948 (+ 217us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 18:34:42.577353 (+ 405us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 18:34:42.577443 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0806 18:34:42.577515 (+ 72us) server_negotiation.cc:299] Negotiation successful
0806 18:34:42.577560 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":287}
I20260806 18:34:42.580209 27531 raft_consensus.cc:1261] T 06184b09bc26427e8636c94069d9c309 P a7dbdd68b24540cda5ada7e1e0d73ccd [term 1 FOLLOWER]: Refusing update from remote peer a5630d59c6324f3b87d9d2a1f4024c45: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 18:34:42.580446 27809 raft_consensus.cc:1261] T 06184b09bc26427e8636c94069d9c309 P 402d0dd8fe1f4b4385b0b94ba213aaa6 [term 1 FOLLOWER]: Refusing update from remote peer a5630d59c6324f3b87d9d2a1f4024c45: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 18:34:42.580753 27970 consensus_queue.cc:1035] T 06184b09bc26427e8636c94069d9c309 P a5630d59c6324f3b87d9d2a1f4024c45 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a7dbdd68b24540cda5ada7e1e0d73ccd" member_type: VOTER last_known_addr { host: "127.19.194.1" port: 33871 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 18:34:42.581017 27963 consensus_queue.cc:1035] T 06184b09bc26427e8636c94069d9c309 P a5630d59c6324f3b87d9d2a1f4024c45 [LEADER]: Connected to new peer: Peer: permanent_uuid: "402d0dd8fe1f4b4385b0b94ba213aaa6" member_type: VOTER last_known_addr { host: "127.19.194.3" port: 42189 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 18:34:42.583488 27939 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:34:42.583132 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.194.62:46737 (local address 127.19.194.1:45673)
0806 18:34:42.583316 (+ 184us) negotiation.cc:107] Waiting for socket to connect
0806 18:34:42.583400 (+ 84us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.194.62:46737: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":94}
I20260806 18:34:42.585438 27973 mvcc.cc:205] Tried to move back new op lower bound from 7315625093443747840 to 7315625093382610944. Current Snapshot: MvccSnapshot[applied={T|T < 7315625093443747840 or (T in {7315625093443747840})}]
I20260806 18:34:42.588354 27940 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:34:42.588046 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.194.62:46737 (local address 127.19.194.3:56885)
0806 18:34:42.588196 (+ 150us) negotiation.cc:107] Waiting for socket to connect
0806 18:34:42.588279 (+ 83us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.194.62:46737: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":86}
I20260806 18:34:42.588915 27858 heartbeater.cc:503] Master 127.19.194.61:39189 was elected leader, sending a full tablet report...
I20260806 18:34:42.589267 27977 mvcc.cc:205] Tried to move back new op lower bound from 7315625093443747840 to 7315625093382610944. Current Snapshot: MvccSnapshot[applied={T|T < 7315625093443747840}]
I20260806 18:34:42.589382 27939 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:34:42.588955 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.194.62:46737 (local address 127.19.194.1:59267)
0806 18:34:42.589251 (+ 296us) negotiation.cc:107] Waiting for socket to connect
0806 18:34:42.589308 (+ 57us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.194.62:46737: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":221}
I20260806 18:34:42.589581 27969 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:34:42.585133 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.194.62:46737 (local address 127.19.194.2:57737)
0806 18:34:42.589452 (+ 4319us) negotiation.cc:107] Waiting for socket to connect
0806 18:34:42.589509 (+ 57us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.194.62:46737: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":4248}
I20260806 18:34:42.590210 27941 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:34:42.589950 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.194.62:46737 (local address 127.19.194.2:53785)
0806 18:34:42.590104 (+ 154us) negotiation.cc:107] Waiting for socket to connect
0806 18:34:42.590153 (+ 49us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.194.62:46737: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":95}
I20260806 18:34:42.594159 27940 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:34:42.593874 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.194.62:46737 (local address 127.19.194.3:52659)
0806 18:34:42.594024 (+ 150us) negotiation.cc:107] Waiting for socket to connect
0806 18:34:42.594073 (+ 49us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.194.62:46737: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":80}
I20260806 18:34:42.594408 27199 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:37874:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 18:34:42.594566 27199 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:37874:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 18:34:42.598716 27199 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 9a38e245bcc34630ba3cc6fb6ca11cb3: Sending DeleteTablet for 3 replicas of tablet 06184b09bc26427e8636c94069d9c309
I20260806 18:34:42.599427 27789 tablet_service.cc:1514] Processing DeleteTablet for tablet 06184b09bc26427e8636c94069d9c309 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 18:34:42 UTC) from {username='oryx', principal='oryx/127.19.194.61@KRBTEST.COM'} at 127.0.0.1:35488
I20260806 18:34:42.599444 27511 tablet_service.cc:1514] Processing DeleteTablet for tablet 06184b09bc26427e8636c94069d9c309 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 18:34:42 UTC) from {username='oryx', principal='oryx/127.19.194.61@KRBTEST.COM'} at 127.0.0.1:35148
I20260806 18:34:42.599807 27985 tablet_replica.cc:330] T 06184b09bc26427e8636c94069d9c309 P a7dbdd68b24540cda5ada7e1e0d73ccd: stopping tablet replica
I20260806 18:34:42.599931 27651 tablet_service.cc:1514] Processing DeleteTablet for tablet 06184b09bc26427e8636c94069d9c309 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 18:34:42 UTC) from {username='oryx', principal='oryx/127.19.194.61@KRBTEST.COM'} at 127.0.0.1:40534
I20260806 18:34:42.599967 27985 raft_consensus.cc:2229] T 06184b09bc26427e8636c94069d9c309 P a7dbdd68b24540cda5ada7e1e0d73ccd [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 18:34:42.600292 27985 raft_consensus.cc:2258] T 06184b09bc26427e8636c94069d9c309 P a7dbdd68b24540cda5ada7e1e0d73ccd [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 18:34:42.600371 20232 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJW69tn/build/debug/bin/kudu with pid 27355
I20260806 18:34:42.601924 27984 tablet_replica.cc:330] T 06184b09bc26427e8636c94069d9c309 P 402d0dd8fe1f4b4385b0b94ba213aaa6: stopping tablet replica
I20260806 18:34:42.602137 27984 raft_consensus.cc:2229] T 06184b09bc26427e8636c94069d9c309 P 402d0dd8fe1f4b4385b0b94ba213aaa6 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 18:34:42.602330 27984 raft_consensus.cc:2258] T 06184b09bc26427e8636c94069d9c309 P 402d0dd8fe1f4b4385b0b94ba213aaa6 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 18:34:42.602727 27986 tablet_replica.cc:330] T 06184b09bc26427e8636c94069d9c309 P a5630d59c6324f3b87d9d2a1f4024c45: stopping tablet replica
I20260806 18:34:42.602909 27986 raft_consensus.cc:2229] T 06184b09bc26427e8636c94069d9c309 P a5630d59c6324f3b87d9d2a1f4024c45 [term 1 LEADER]: Raft consensus shutting down.
I20260806 18:34:42.603233 27986 raft_consensus.cc:2258] T 06184b09bc26427e8636c94069d9c309 P a5630d59c6324f3b87d9d2a1f4024c45 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 18:34:42.604133 27986 ts_tablet_manager.cc:1905] T 06184b09bc26427e8636c94069d9c309 P a5630d59c6324f3b87d9d2a1f4024c45: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 18:34:42.607056 27986 ts_tablet_manager.cc:1918] T 06184b09bc26427e8636c94069d9c309 P a5630d59c6324f3b87d9d2a1f4024c45: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 18:34:42.607113 27986 log.cc:1196] T 06184b09bc26427e8636c94069d9c309 P a5630d59c6324f3b87d9d2a1f4024c45: Deleting WAL directory at /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786041246094603-20232-0/minicluster-data/ts-1/wal/wals/06184b09bc26427e8636c94069d9c309
I20260806 18:34:42.607479 27986 ts_tablet_manager.cc:1939] T 06184b09bc26427e8636c94069d9c309 P a5630d59c6324f3b87d9d2a1f4024c45: Deleting consensus metadata
I20260806 18:34:42.607839 27984 ts_tablet_manager.cc:1905] T 06184b09bc26427e8636c94069d9c309 P 402d0dd8fe1f4b4385b0b94ba213aaa6: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 18:34:42.608211 27194 catalog_manager.cc:4841] TS a5630d59c6324f3b87d9d2a1f4024c45 (127.19.194.2:44715): tablet 06184b09bc26427e8636c94069d9c309 (table test-table [id=c8d1a5008685469b85bf57eff0b0b0a7]) successfully deleted
I20260806 18:34:42.610054 27984 ts_tablet_manager.cc:1918] T 06184b09bc26427e8636c94069d9c309 P 402d0dd8fe1f4b4385b0b94ba213aaa6: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 18:34:42.610128 27984 log.cc:1196] T 06184b09bc26427e8636c94069d9c309 P 402d0dd8fe1f4b4385b0b94ba213aaa6: Deleting WAL directory at /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786041246094603-20232-0/minicluster-data/ts-2/wal/wals/06184b09bc26427e8636c94069d9c309
I20260806 18:34:42.610440 27984 ts_tablet_manager.cc:1939] T 06184b09bc26427e8636c94069d9c309 P 402d0dd8fe1f4b4385b0b94ba213aaa6: Deleting consensus metadata
I20260806 18:34:42.610999 27955 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:34:42.610717 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.194.1:33871 (local address 127.0.0.1:35158)
0806 18:34:42.610865 (+ 148us) negotiation.cc:107] Waiting for socket to connect
0806 18:34:42.610921 (+ 56us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.194.1:33871: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":57}
W20260806 18:34:42.611204 27191 catalog_manager.cc:4584] TS a7dbdd68b24540cda5ada7e1e0d73ccd (127.19.194.1:33871): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 06184b09bc26427e8636c94069d9c309: Network error: Client connection negotiation failed: client connection to 127.19.194.1:33871: connect: Connection refused (error 111)
I20260806 18:34:42.611380 27193 catalog_manager.cc:4841] TS 402d0dd8fe1f4b4385b0b94ba213aaa6 (127.19.194.3:42189): tablet 06184b09bc26427e8636c94069d9c309 (table test-table [id=c8d1a5008685469b85bf57eff0b0b0a7]) successfully deleted
I20260806 18:34:42.611624 20232 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJW69tn/build/debug/bin/kudu with pid 27586
I20260806 18:34:42.618543 20232 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJW69tn/build/debug/bin/kudu with pid 27725
I20260806 18:34:42.626448 20232 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJW69tn/build/debug/bin/kudu with pid 27070
I20260806 18:34:42.633155 20232 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJW69tn/build/debug/bin/kudu with pid 27179
2026-08-06T18:34:42Z chronyd exiting
[ OK ] SecurityITest.TestNonDefaultPrincipalMultipleMaster (7101 ms)
[ RUN ] SecurityITest.TestNonDefaultPrincipal
Loading random data
Initializing database '/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786041246094603-20232-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 18:34:42 dist-test-slave-1xrh krb5kdc[27989](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 18:34:42 dist-test-slave-1xrh krb5kdc[27989](info): set up 2 sockets
Aug 06 18:34:42 dist-test-slave-1xrh krb5kdc[27989](info): commencing operation
krb5kdc: starting...
W20260806 18:34:43.704900 20232 mini_kdc.cc:121] Time spent starting KDC: real 1.047s user 0.003s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786041246094603-20232-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786041246094603-20232-0/krb5kdc/test-user.keytab.
Aug 06 18:34:43 dist-test-slave-1xrh krb5kdc[27989](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041283, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T18:34:43Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T18:34:43Z Disabled control of system clock
WARNING: no policy specified for oryx/127.19.194.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.19.194.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.19.194.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786041246094603-20232-0/krb5kdc/oryx_127.19.194.62.keytab.
Entry for principal oryx/127.19.194.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786041246094603-20232-0/krb5kdc/oryx_127.19.194.62.keytab.
I20260806 18:34:43.825649 20232 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJW69tn/build/debug/bin/kudu
/tmp/dist-test-taskJW69tn/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786041246094603-20232-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786041246094603-20232-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786041246094603-20232-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786041246094603-20232-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.19.194.62:33607
--webserver_interface=127.19.194.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786041246094603-20232-0/krb5kdc/oryx_127.19.194.62.keytab
--principal=oryx/127.19.194.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.19.194.20:36279
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.19.194.62:33607
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786041246094603-20232-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786041246094603-20232-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786041246094603-20232-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 18:34:43.933861 28003 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 18:34:43.934145 28003 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 18:34:43.934218 28003 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 18:34:43.938314 28003 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 18:34:43.938397 28003 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260806 18:34:43.938431 28003 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 18:34:43.938468 28003 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 18:34:43.938500 28003 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 18:34:43.938544 28003 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 18:34:43.943758 28003 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.194.20:36279
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786041246094603-20232-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786041246094603-20232-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.19.194.62:33607
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786041246094603-20232-0/krb5kdc/oryx_127.19.194.62.keytab
--principal=oryx/127.19.194.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.19.194.62:33607
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786041246094603-20232-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.19.194.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28003
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786041246094603-20232-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 17:59:13 UTC on ccycloud.quasar-wvsnzx.root.comops.site
build id 4571028
I20260806 18:34:43.944746 28003 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 18:34:43.945726 28003 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 18:34:43.953946 28008 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:34:43.953954 28011 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:34:43.953959 28009 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 18:34:43.954259 28003 server_base.cc:1034] running on GCE node
I20260806 18:34:43.954800 28003 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 18:34:43.955940 28003 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 18:34:43.957130 28003 hybrid_clock.cc:648] HybridClock initialized: now 1786041283957112 us; error 30 us; skew 500 ppm
Aug 06 18:34:43 dist-test-slave-1xrh krb5kdc[27989](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041283, etypes {rep=17 tkt=17 ses=17}, oryx/127.19.194.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 18:34:43.965785 28003 init.cc:377] Logged in from keytab as oryx/127.19.194.62@KRBTEST.COM (short username oryx)
I20260806 18:34:43.966684 28003 webserver.cc:460] Webserver started at http://127.19.194.62:38807/ using document root <none> and password file <none>
I20260806 18:34:43.967314 28003 fs_manager.cc:359] Metadata directory not provided
I20260806 18:34:43.967411 28003 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 18:34:43.967656 28003 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 18:34:43.969622 28003 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786041246094603-20232-0/minicluster-data/master-0/data/instance:
uuid: "d0282f2efb864823bbf90a21992ab39d"
format_stamp: "Formatted at 2026-08-06 18:34:43 on dist-test-slave-1xrh"
server_key: "7090c410234407a180c148104dc35bf6"
server_key_iv: "613c838ee278fdc8356630b2a32da5f7"
server_key_version: "encryptionkey@0"
I20260806 18:34:43.970191 28003 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786041246094603-20232-0/minicluster-data/master-0/wal/instance:
uuid: "d0282f2efb864823bbf90a21992ab39d"
format_stamp: "Formatted at 2026-08-06 18:34:43 on dist-test-slave-1xrh"
server_key: "7090c410234407a180c148104dc35bf6"
server_key_iv: "613c838ee278fdc8356630b2a32da5f7"
server_key_version: "encryptionkey@0"
I20260806 18:34:43.973812 28003 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.003s
I20260806 18:34:43.976135 28018 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 18:34:43.977247 28003 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260806 18:34:43.977399 28003 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786041246094603-20232-0/minicluster-data/master-0/data,/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786041246094603-20232-0/minicluster-data/master-0/wal
uuid: "d0282f2efb864823bbf90a21992ab39d"
format_stamp: "Formatted at 2026-08-06 18:34:43 on dist-test-slave-1xrh"
server_key: "7090c410234407a180c148104dc35bf6"
server_key_iv: "613c838ee278fdc8356630b2a32da5f7"
server_key_version: "encryptionkey@0"
I20260806 18:34:43.977506 28003 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786041246094603-20232-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786041246094603-20232-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786041246094603-20232-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 18:34:43.987317 28003 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 18:34:43.989948 28003 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 18:34:43.990161 28003 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 18:34:43.998412 28003 rpc_server.cc:307] RPC server started. Bound to: 127.19.194.62:33607
I20260806 18:34:43.998441 28081 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.194.62:33607 every 8 connection(s)
I20260806 18:34:43.999346 28003 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786041246094603-20232-0/minicluster-data/master-0/data/info.pb
I20260806 18:34:44.001564 20232 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJW69tn/build/debug/bin/kudu as pid 28003
I20260806 18:34:44.001667 20232 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786041246094603-20232-0/minicluster-data/master-0/wal/instance
I20260806 18:34:44.002003 20232 external_mini_cluster.cc:1428] Setting key 5abaee3a096e2d8baaeb623a67e971dc
I20260806 18:34:44.002079 28082 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 18:34:44.009452 28082 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d0282f2efb864823bbf90a21992ab39d: Bootstrap starting.
Aug 06 18:34:44 dist-test-slave-1xrh krb5kdc[27989](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041283, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.19.194.62@KRBTEST.COM
I20260806 18:34:44.012140 28082 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P d0282f2efb864823bbf90a21992ab39d: Neither blocks nor log segments found. Creating new log.
I20260806 18:34:44.013015 28082 log.cc:824] T 00000000000000000000000000000000 P d0282f2efb864823bbf90a21992ab39d: Log is configured to *not* fsync() on all Append() calls
I20260806 18:34:44.015179 28082 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d0282f2efb864823bbf90a21992ab39d: No bootstrap required, opened a new log
I20260806 18:34:44.017803 28085 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:34:44.003499 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46764 (local address 127.19.194.62:33607)
0806 18:34:44.003962 (+ 463us) server_negotiation.cc:206] Beginning negotiation
0806 18:34:44.003983 (+ 21us) server_negotiation.cc:399] Waiting for connection header
0806 18:34:44.004024 (+ 41us) server_negotiation.cc:407] Connection header received
0806 18:34:44.004909 (+ 885us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:34:44.004934 (+ 25us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:34:44.005343 (+ 409us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:34:44.005733 (+ 390us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:34:44.006457 (+ 724us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:34:44.007939 (+ 1482us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:34:44.008906 (+ 967us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:34:44.009584 (+ 678us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:34:44.009737 (+ 153us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:34:44.012567 (+ 2830us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:34:44.012573 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:34:44.012585 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 18:34:44.012612 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:34:44.015001 (+ 2389us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:34:44.015520 (+ 519us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:34:44.015525 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:34:44.015531 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:34:44.015647 (+ 116us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:34:44.015949 (+ 302us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:34:44.015951 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:34:44.015952 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:34:44.016339 (+ 387us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:34:44.016478 (+ 139us) server_negotiation.cc:1035] Waiting for connection context
0806 18:34:44.016779 (+ 301us) server_negotiation.cc:299] Negotiation successful
0806 18:34:44.017088 (+ 309us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":314,"thread_start_us":118,"threads_started":1}
I20260806 18:34:44.018512 28082 raft_consensus.cc:348] T 00000000000000000000000000000000 P d0282f2efb864823bbf90a21992ab39d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d0282f2efb864823bbf90a21992ab39d" member_type: VOTER last_known_addr { host: "127.19.194.62" port: 33607 } }
I20260806 18:34:44.018752 28082 raft_consensus.cc:374] T 00000000000000000000000000000000 P d0282f2efb864823bbf90a21992ab39d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 18:34:44.018831 28082 raft_consensus.cc:729] T 00000000000000000000000000000000 P d0282f2efb864823bbf90a21992ab39d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d0282f2efb864823bbf90a21992ab39d, State: Initialized, Role: FOLLOWER
I20260806 18:34:44.019299 28082 consensus_queue.cc:260] T 00000000000000000000000000000000 P d0282f2efb864823bbf90a21992ab39d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d0282f2efb864823bbf90a21992ab39d" member_type: VOTER last_known_addr { host: "127.19.194.62" port: 33607 } }
I20260806 18:34:44.019454 28082 raft_consensus.cc:388] T 00000000000000000000000000000000 P d0282f2efb864823bbf90a21992ab39d [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 18:34:44.019536 28082 raft_consensus.cc:482] T 00000000000000000000000000000000 P d0282f2efb864823bbf90a21992ab39d [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 18:34:44.019626 28082 raft_consensus.cc:3037] T 00000000000000000000000000000000 P d0282f2efb864823bbf90a21992ab39d [term 0 FOLLOWER]: Advancing to term 1
I20260806 18:34:44.020629 28082 raft_consensus.cc:504] T 00000000000000000000000000000000 P d0282f2efb864823bbf90a21992ab39d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d0282f2efb864823bbf90a21992ab39d" member_type: VOTER last_known_addr { host: "127.19.194.62" port: 33607 } }
I20260806 18:34:44.020908 28082 leader_election.cc:302] T 00000000000000000000000000000000 P d0282f2efb864823bbf90a21992ab39d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: d0282f2efb864823bbf90a21992ab39d; no voters:
I20260806 18:34:44.021139 28082 leader_election.cc:288] T 00000000000000000000000000000000 P d0282f2efb864823bbf90a21992ab39d [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 18:34:44.021288 28087 raft_consensus.cc:2781] T 00000000000000000000000000000000 P d0282f2efb864823bbf90a21992ab39d [term 1 FOLLOWER]: Leader election won for term 1
I20260806 18:34:44.021512 28087 raft_consensus.cc:686] T 00000000000000000000000000000000 P d0282f2efb864823bbf90a21992ab39d [term 1 LEADER]: Becoming Leader. State: Replica: d0282f2efb864823bbf90a21992ab39d, State: Running, Role: LEADER
I20260806 18:34:44.021867 28087 consensus_queue.cc:237] T 00000000000000000000000000000000 P d0282f2efb864823bbf90a21992ab39d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d0282f2efb864823bbf90a21992ab39d" member_type: VOTER last_known_addr { host: "127.19.194.62" port: 33607 } }
I20260806 18:34:44.022301 28082 sys_catalog.cc:564] T 00000000000000000000000000000000 P d0282f2efb864823bbf90a21992ab39d [sys.catalog]: configured and running, proceeding with master startup.
I20260806 18:34:44.023468 28089 sys_catalog.cc:455] T 00000000000000000000000000000000 P d0282f2efb864823bbf90a21992ab39d [sys.catalog]: SysCatalogTable state changed. Reason: New leader d0282f2efb864823bbf90a21992ab39d. Latest consensus state: current_term: 1 leader_uuid: "d0282f2efb864823bbf90a21992ab39d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d0282f2efb864823bbf90a21992ab39d" member_type: VOTER last_known_addr { host: "127.19.194.62" port: 33607 } } }
I20260806 18:34:44.023592 28089 sys_catalog.cc:458] T 00000000000000000000000000000000 P d0282f2efb864823bbf90a21992ab39d [sys.catalog]: This master's current role is: LEADER
I20260806 18:34:44.023936 28088 sys_catalog.cc:455] T 00000000000000000000000000000000 P d0282f2efb864823bbf90a21992ab39d [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "d0282f2efb864823bbf90a21992ab39d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d0282f2efb864823bbf90a21992ab39d" member_type: VOTER last_known_addr { host: "127.19.194.62" port: 33607 } } }
I20260806 18:34:44.024009 28096 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 18:34:44.024032 28088 sys_catalog.cc:458] T 00000000000000000000000000000000 P d0282f2efb864823bbf90a21992ab39d [sys.catalog]: This master's current role is: LEADER
I20260806 18:34:44.026741 28096 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 18:34:44.032006 28096 catalog_manager.cc:1347] Generated new cluster ID: b8480a8aa24c429597d99bbcd00b8507
I20260806 18:34:44.032071 28096 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 18:34:44.037477 28096 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 18:34:44.038631 28096 catalog_manager.cc:1504] Loading token signing keys...
I20260806 18:34:44.045774 28096 catalog_manager.cc:5868] T 00000000000000000000000000000000 P d0282f2efb864823bbf90a21992ab39d: Generated new TSK 0
I20260806 18:34:44.046458 28096 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for oryx/127.19.194.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.19.194.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.19.194.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786041246094603-20232-0/krb5kdc/oryx_127.19.194.1.keytab.
Entry for principal oryx/127.19.194.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786041246094603-20232-0/krb5kdc/oryx_127.19.194.1.keytab.
I20260806 18:34:44.077998 20232 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJW69tn/build/debug/bin/kudu
/tmp/dist-test-taskJW69tn/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786041246094603-20232-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786041246094603-20232-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786041246094603-20232-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786041246094603-20232-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.19.194.1:0
--local_ip_for_outbound_sockets=127.19.194.1
--webserver_interface=127.19.194.1
--webserver_port=0
--tserver_master_addrs=127.19.194.62:33607
--keytab_file=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786041246094603-20232-0/krb5kdc/oryx_127.19.194.1.keytab
--principal=oryx/127.19.194.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.19.194.20:36279
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786041246094603-20232-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786041246094603-20232-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786041246094603-20232-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 18:34:44.187958 28108 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 18:34:44.188194 28108 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 18:34:44.188235 28108 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 18:34:44.192128 28108 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 18:34:44.192175 28108 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 18:34:44.192214 28108 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260806 18:34:44.192252 28108 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.19.194.1
I20260806 18:34:44.197309 28108 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.194.20:36279
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786041246094603-20232-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786041246094603-20232-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786041246094603-20232-0/krb5kdc/oryx_127.19.194.1.keytab
--principal=oryx/127.19.194.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.19.194.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786041246094603-20232-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.19.194.1
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.19.194.62:33607
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28108
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.19.194.1
--log_dir=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786041246094603-20232-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 17:59:13 UTC on ccycloud.quasar-wvsnzx.root.comops.site
build id 4571028
I20260806 18:34:44.198273 28108 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 18:34:44.199219 28108 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 18:34:44.208771 28116 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:34:44.208801 28113 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:34:44.208811 28114 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 18:34:44.208946 28108 server_base.cc:1034] running on GCE node
I20260806 18:34:44.209522 28108 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 18:34:44.210132 28108 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 18:34:44.211292 28108 hybrid_clock.cc:648] HybridClock initialized: now 1786041284211277 us; error 24 us; skew 500 ppm
Aug 06 18:34:44 dist-test-slave-1xrh krb5kdc[27989](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041284, etypes {rep=17 tkt=17 ses=17}, oryx/127.19.194.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 18:34:44.214148 28108 init.cc:377] Logged in from keytab as oryx/127.19.194.1@KRBTEST.COM (short username oryx)
I20260806 18:34:44.214962 28108 webserver.cc:460] Webserver started at http://127.19.194.1:36875/ using document root <none> and password file <none>
I20260806 18:34:44.215562 28108 fs_manager.cc:359] Metadata directory not provided
I20260806 18:34:44.215629 28108 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 18:34:44.215860 28108 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 18:34:44.217808 28108 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786041246094603-20232-0/minicluster-data/ts-0/data/instance:
uuid: "885703862b27424890515f701c83aa68"
format_stamp: "Formatted at 2026-08-06 18:34:44 on dist-test-slave-1xrh"
server_key: "66f05153da854c3fbd74d1137a3fb028"
server_key_iv: "289da103a403afb019ecf43d8b7be9df"
server_key_version: "encryptionkey@0"
I20260806 18:34:44.218384 28108 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786041246094603-20232-0/minicluster-data/ts-0/wal/instance:
uuid: "885703862b27424890515f701c83aa68"
format_stamp: "Formatted at 2026-08-06 18:34:44 on dist-test-slave-1xrh"
server_key: "66f05153da854c3fbd74d1137a3fb028"
server_key_iv: "289da103a403afb019ecf43d8b7be9df"
server_key_version: "encryptionkey@0"
I20260806 18:34:44.222015 28108 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.006s sys 0.000s
I20260806 18:34:44.224680 28123 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 18:34:44.225841 28108 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260806 18:34:44.225965 28108 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786041246094603-20232-0/minicluster-data/ts-0/data,/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786041246094603-20232-0/minicluster-data/ts-0/wal
uuid: "885703862b27424890515f701c83aa68"
format_stamp: "Formatted at 2026-08-06 18:34:44 on dist-test-slave-1xrh"
server_key: "66f05153da854c3fbd74d1137a3fb028"
server_key_iv: "289da103a403afb019ecf43d8b7be9df"
server_key_version: "encryptionkey@0"
I20260806 18:34:44.226075 28108 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786041246094603-20232-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786041246094603-20232-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786041246094603-20232-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 18:34:44.241065 28108 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 18:34:44.243860 28108 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 18:34:44.244058 28108 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 18:34:44.245409 28108 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 18:34:44.245463 28108 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 18:34:44.245532 28108 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 18:34:44.245568 28108 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 06 18:34:44 dist-test-slave-1xrh krb5kdc[27989](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041284, etypes {rep=17 tkt=17 ses=17}, oryx/127.19.194.1@KRBTEST.COM for oryx/127.19.194.62@KRBTEST.COM
I20260806 18:34:44.256956 28108 rpc_server.cc:307] RPC server started. Bound to: 127.19.194.1:34859
I20260806 18:34:44.257647 28238 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.194.1:34859 every 8 connection(s)
I20260806 18:34:44.258062 28108 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786041246094603-20232-0/minicluster-data/ts-0/data/info.pb
I20260806 18:34:44.259992 28085 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:34:44.246297 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.194.1:50111 (local address 127.19.194.62:33607)
0806 18:34:44.246426 (+ 129us) server_negotiation.cc:206] Beginning negotiation
0806 18:34:44.246430 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 18:34:44.247531 (+ 1101us) server_negotiation.cc:407] Connection header received
0806 18:34:44.248601 (+ 1070us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:34:44.248607 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:34:44.248695 (+ 88us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:34:44.248762 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:34:44.250248 (+ 1486us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:34:44.250949 (+ 701us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:34:44.252883 (+ 1934us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:34:44.253252 (+ 369us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:34:44.253345 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:34:44.256276 (+ 2931us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:34:44.256280 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:34:44.256282 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 18:34:44.256310 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:34:44.257893 (+ 1583us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:34:44.258522 (+ 629us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:34:44.258525 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:34:44.258527 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:34:44.258623 (+ 96us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:34:44.259016 (+ 393us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:34:44.259019 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:34:44.259020 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:34:44.259286 (+ 266us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:34:44.259367 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0806 18:34:44.259772 (+ 405us) server_negotiation.cc:299] Negotiation successful
0806 18:34:44.259874 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260806 18:34:44.260680 28132 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:34:44.246815 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.194.62:33607 (local address 127.19.194.1:50111)
0806 18:34:44.247378 (+ 563us) negotiation.cc:107] Waiting for socket to connect
0806 18:34:44.247419 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0806 18:34:44.248331 (+ 912us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:34:44.249190 (+ 859us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:34:44.249196 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:34:44.249566 (+ 370us) client_negotiation.cc:189] Negotiated authn=SASL
0806 18:34:44.250095 (+ 529us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:34:44.250110 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:34:44.251112 (+ 1002us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:34:44.251117 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:34:44.252725 (+ 1608us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:34:44.252740 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:34:44.253412 (+ 672us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:34:44.253421 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:34:44.253667 (+ 246us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:34:44.254202 (+ 535us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 18:34:44.254230 (+ 28us) client_negotiation.cc:656] calling sasl_client_start()
0806 18:34:44.256106 (+ 1876us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 18:34:44.258037 (+ 1931us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:34:44.258043 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:34:44.258057 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:34:44.258385 (+ 328us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:34:44.258746 (+ 361us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:34:44.258750 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:34:44.258752 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:34:44.258905 (+ 153us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:34:44.259369 (+ 464us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 18:34:44.259373 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 18:34:44.259566 (+ 193us) client_negotiation.cc:769] Sending connection context
0806 18:34:44.259702 (+ 136us) client_negotiation.cc:240] Negotiation successful
0806 18:34:44.259913 (+ 211us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":384,"thread_start_us":172,"threads_started":1}
I20260806 18:34:44.261837 28239 heartbeater.cc:347] Connected to a master server at 127.19.194.62:33607
I20260806 18:34:44.261993 28239 heartbeater.cc:464] Registering TS with master...
I20260806 18:34:44.262440 28239 heartbeater.cc:511] Master 127.19.194.62:33607 requested a full tablet report, sending...
I20260806 18:34:44.264045 28034 ts_manager.cc:194] Registered new tserver with Master: 885703862b27424890515f701c83aa68 (127.19.194.1:34859)
I20260806 18:34:44.264326 20232 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJW69tn/build/debug/bin/kudu as pid 28108
I20260806 18:34:44.264463 20232 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786041246094603-20232-0/minicluster-data/ts-0/wal/instance
I20260806 18:34:44.264765 20232 external_mini_cluster.cc:1428] Setting key 4cda7b79f0af6615975efb3950159a02
I20260806 18:34:44.265506 28034 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.19.194.1@KRBTEST.COM'} at 127.19.194.1:50111
I20260806 18:34:44.273327 28085 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:34:44.264036 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.194.1:56179 (local address 127.19.194.62:33607)
0806 18:34:44.265184 (+ 1148us) server_negotiation.cc:206] Beginning negotiation
0806 18:34:44.265188 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 18:34:44.265432 (+ 244us) server_negotiation.cc:407] Connection header received
0806 18:34:44.265612 (+ 180us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:34:44.265617 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:34:44.265664 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:34:44.265746 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:34:44.267258 (+ 1512us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:34:44.267742 (+ 484us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:34:44.268556 (+ 814us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:34:44.268774 (+ 218us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:34:44.268883 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:34:44.270150 (+ 1267us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:34:44.270153 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:34:44.270155 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 18:34:44.270177 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:34:44.271629 (+ 1452us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:34:44.272151 (+ 522us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:34:44.272153 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:34:44.272155 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:34:44.272225 (+ 70us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:34:44.272545 (+ 320us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:34:44.272547 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:34:44.272548 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:34:44.272700 (+ 152us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:34:44.272774 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0806 18:34:44.273127 (+ 353us) server_negotiation.cc:299] Negotiation successful
0806 18:34:44.273217 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1077}
I20260806 18:34:44.273455 28245 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:34:44.263957 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.194.62:33607 (local address 127.19.194.1:56179)
0806 18:34:44.265351 (+ 1394us) negotiation.cc:107] Waiting for socket to connect
0806 18:34:44.265368 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0806 18:34:44.265499 (+ 131us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:34:44.265764 (+ 265us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:34:44.265767 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:34:44.266009 (+ 242us) client_negotiation.cc:189] Negotiated authn=SASL
0806 18:34:44.266141 (+ 132us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:34:44.266149 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:34:44.267875 (+ 1726us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:34:44.267878 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:34:44.268425 (+ 547us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:34:44.268433 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:34:44.268898 (+ 465us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:34:44.268901 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:34:44.269002 (+ 101us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:34:44.269507 (+ 505us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 18:34:44.269526 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0806 18:34:44.270006 (+ 480us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 18:34:44.271745 (+ 1739us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:34:44.271751 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:34:44.271756 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:34:44.272048 (+ 292us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:34:44.272323 (+ 275us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:34:44.272330 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:34:44.272334 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:34:44.272445 (+ 111us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:34:44.272803 (+ 358us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 18:34:44.272808 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 18:34:44.272929 (+ 121us) client_negotiation.cc:769] Sending connection context
0806 18:34:44.273069 (+ 140us) client_negotiation.cc:240] Negotiation successful
0806 18:34:44.273230 (+ 161us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1316,"thread_start_us":104,"threads_started":1}
WARNING: no policy specified for oryx/127.19.194.2@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.19.194.2@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.19.194.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786041246094603-20232-0/krb5kdc/oryx_127.19.194.2.keytab.
Entry for principal oryx/127.19.194.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786041246094603-20232-0/krb5kdc/oryx_127.19.194.2.keytab.
I20260806 18:34:44.294596 20232 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJW69tn/build/debug/bin/kudu
/tmp/dist-test-taskJW69tn/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786041246094603-20232-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786041246094603-20232-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786041246094603-20232-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786041246094603-20232-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.19.194.2:0
--local_ip_for_outbound_sockets=127.19.194.2
--webserver_interface=127.19.194.2
--webserver_port=0
--tserver_master_addrs=127.19.194.62:33607
--keytab_file=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786041246094603-20232-0/krb5kdc/oryx_127.19.194.2.keytab
--principal=oryx/127.19.194.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.19.194.20:36279
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786041246094603-20232-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786041246094603-20232-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786041246094603-20232-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 18:34:44.404903 28248 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 18:34:44.405141 28248 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 18:34:44.405182 28248 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 18:34:44.409005 28248 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 18:34:44.409052 28248 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 18:34:44.409085 28248 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260806 18:34:44.409113 28248 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.19.194.2
I20260806 18:34:44.414191 28248 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.194.20:36279
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786041246094603-20232-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786041246094603-20232-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786041246094603-20232-0/krb5kdc/oryx_127.19.194.2.keytab
--principal=oryx/127.19.194.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.19.194.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786041246094603-20232-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.19.194.2
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.19.194.62:33607
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28248
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.19.194.2
--log_dir=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786041246094603-20232-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 17:59:13 UTC on ccycloud.quasar-wvsnzx.root.comops.site
build id 4571028
I20260806 18:34:44.415160 28248 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 18:34:44.416213 28248 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 18:34:44.426090 28256 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:34:44.426151 28254 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 18:34:44.426177 28248 server_base.cc:1034] running on GCE node
W20260806 18:34:44.426090 28253 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 18:34:44.426777 28248 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 18:34:44.427480 28248 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 18:34:44.428649 28248 hybrid_clock.cc:648] HybridClock initialized: now 1786041284428635 us; error 25 us; skew 500 ppm
Aug 06 18:34:44 dist-test-slave-1xrh krb5kdc[27989](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041284, etypes {rep=17 tkt=17 ses=17}, oryx/127.19.194.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 18:34:44.431583 28248 init.cc:377] Logged in from keytab as oryx/127.19.194.2@KRBTEST.COM (short username oryx)
I20260806 18:34:44.432459 28248 webserver.cc:460] Webserver started at http://127.19.194.2:46737/ using document root <none> and password file <none>
I20260806 18:34:44.433007 28248 fs_manager.cc:359] Metadata directory not provided
I20260806 18:34:44.433112 28248 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 18:34:44.433341 28248 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 18:34:44.435422 28248 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786041246094603-20232-0/minicluster-data/ts-1/data/instance:
uuid: "4575246688ab47bdb5abf0da5c702767"
format_stamp: "Formatted at 2026-08-06 18:34:44 on dist-test-slave-1xrh"
server_key: "6aa68bbce2bb1d9e7db56b0a0634e321"
server_key_iv: "f684d2ac8fd6530c44ecfc152f39026b"
server_key_version: "encryptionkey@0"
I20260806 18:34:44.436002 28248 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786041246094603-20232-0/minicluster-data/ts-1/wal/instance:
uuid: "4575246688ab47bdb5abf0da5c702767"
format_stamp: "Formatted at 2026-08-06 18:34:44 on dist-test-slave-1xrh"
server_key: "6aa68bbce2bb1d9e7db56b0a0634e321"
server_key_iv: "f684d2ac8fd6530c44ecfc152f39026b"
server_key_version: "encryptionkey@0"
I20260806 18:34:44.439687 28248 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.001s
I20260806 18:34:44.441915 28263 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 18:34:44.442988 28248 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260806 18:34:44.443163 28248 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786041246094603-20232-0/minicluster-data/ts-1/data,/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786041246094603-20232-0/minicluster-data/ts-1/wal
uuid: "4575246688ab47bdb5abf0da5c702767"
format_stamp: "Formatted at 2026-08-06 18:34:44 on dist-test-slave-1xrh"
server_key: "6aa68bbce2bb1d9e7db56b0a0634e321"
server_key_iv: "f684d2ac8fd6530c44ecfc152f39026b"
server_key_version: "encryptionkey@0"
I20260806 18:34:44.443274 28248 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786041246094603-20232-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786041246094603-20232-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786041246094603-20232-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 18:34:44.460875 28248 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 18:34:44.464111 28248 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 18:34:44.464309 28248 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 18:34:44.465651 28248 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 18:34:44.465728 28248 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 18:34:44.465799 28248 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 18:34:44.465848 28248 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 06 18:34:44 dist-test-slave-1xrh krb5kdc[27989](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041284, etypes {rep=17 tkt=17 ses=17}, oryx/127.19.194.2@KRBTEST.COM for oryx/127.19.194.62@KRBTEST.COM
I20260806 18:34:44.477253 28248 rpc_server.cc:307] RPC server started. Bound to: 127.19.194.2:44765
I20260806 18:34:44.477293 28387 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.194.2:44765 every 8 connection(s)
I20260806 18:34:44.478302 28248 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786041246094603-20232-0/minicluster-data/ts-1/data/info.pb
I20260806 18:34:44.479615 28085 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:34:44.466499 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.194.2:46797 (local address 127.19.194.62:33607)
0806 18:34:44.466691 (+ 192us) server_negotiation.cc:206] Beginning negotiation
0806 18:34:44.466697 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 18:34:44.467479 (+ 782us) server_negotiation.cc:407] Connection header received
0806 18:34:44.468394 (+ 915us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:34:44.468401 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:34:44.468466 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:34:44.468585 (+ 119us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:34:44.469739 (+ 1154us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:34:44.470256 (+ 517us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:34:44.471550 (+ 1294us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:34:44.471895 (+ 345us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:34:44.472031 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:34:44.475678 (+ 3647us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:34:44.475685 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:34:44.475691 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 18:34:44.475724 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:34:44.477390 (+ 1666us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:34:44.477947 (+ 557us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:34:44.477949 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:34:44.477950 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:34:44.478028 (+ 78us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:34:44.478413 (+ 385us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:34:44.478417 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:34:44.478418 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:34:44.478602 (+ 184us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:34:44.478758 (+ 156us) server_negotiation.cc:1035] Waiting for connection context
0806 18:34:44.479379 (+ 621us) server_negotiation.cc:299] Negotiation successful
0806 18:34:44.479493 (+ 114us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
I20260806 18:34:44.480365 28272 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:34:44.466822 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.194.62:33607 (local address 127.19.194.2:46797)
0806 18:34:44.467308 (+ 486us) negotiation.cc:107] Waiting for socket to connect
0806 18:34:44.467355 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0806 18:34:44.468190 (+ 835us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:34:44.468774 (+ 584us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:34:44.468782 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:34:44.469153 (+ 371us) client_negotiation.cc:189] Negotiated authn=SASL
0806 18:34:44.469590 (+ 437us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:34:44.469604 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:34:44.470429 (+ 825us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:34:44.470433 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:34:44.471413 (+ 980us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:34:44.471423 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:34:44.472073 (+ 650us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:34:44.472077 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:34:44.472287 (+ 210us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:34:44.472890 (+ 603us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 18:34:44.472906 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0806 18:34:44.475460 (+ 2554us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 18:34:44.477506 (+ 2046us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:34:44.477511 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:34:44.477530 (+ 19us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:34:44.477839 (+ 309us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:34:44.478117 (+ 278us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:34:44.478120 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:34:44.478122 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:34:44.478296 (+ 174us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:34:44.478781 (+ 485us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 18:34:44.478787 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 18:34:44.479066 (+ 279us) client_negotiation.cc:769] Sending connection context
0806 18:34:44.479289 (+ 223us) client_negotiation.cc:240] Negotiation successful
0806 18:34:44.479581 (+ 292us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":274,"thread_start_us":134,"threads_started":1}
I20260806 18:34:44.480808 20232 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJW69tn/build/debug/bin/kudu as pid 28248
I20260806 18:34:44.480985 20232 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786041246094603-20232-0/minicluster-data/ts-1/wal/instance
I20260806 18:34:44.481344 20232 external_mini_cluster.cc:1428] Setting key 408ca196c89137b4579f41202c1ec90b
I20260806 18:34:44.483718 28388 heartbeater.cc:347] Connected to a master server at 127.19.194.62:33607
I20260806 18:34:44.483879 28388 heartbeater.cc:464] Registering TS with master...
I20260806 18:34:44.484520 28388 heartbeater.cc:511] Master 127.19.194.62:33607 requested a full tablet report, sending...
I20260806 18:34:44.485507 28035 ts_manager.cc:194] Registered new tserver with Master: 4575246688ab47bdb5abf0da5c702767 (127.19.194.2:44765)
I20260806 18:34:44.486294 28035 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.19.194.2@KRBTEST.COM'} at 127.19.194.2:46797
I20260806 18:34:44.491060 28085 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:34:44.483948 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.194.2:57257 (local address 127.19.194.62:33607)
0806 18:34:44.484072 (+ 124us) server_negotiation.cc:206] Beginning negotiation
0806 18:34:44.484076 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 18:34:44.484364 (+ 288us) server_negotiation.cc:407] Connection header received
0806 18:34:44.484467 (+ 103us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:34:44.484470 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:34:44.484512 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:34:44.484600 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:34:44.485089 (+ 489us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:34:44.485634 (+ 545us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:34:44.486460 (+ 826us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:34:44.486769 (+ 309us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:34:44.486898 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:34:44.487988 (+ 1090us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:34:44.487991 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:34:44.487992 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 18:34:44.488013 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:34:44.489370 (+ 1357us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:34:44.489871 (+ 501us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:34:44.489873 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:34:44.489875 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:34:44.489944 (+ 69us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:34:44.490259 (+ 315us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:34:44.490261 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:34:44.490262 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:34:44.490446 (+ 184us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:34:44.490531 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0806 18:34:44.490857 (+ 326us) server_negotiation.cc:299] Negotiation successful
0806 18:34:44.490948 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":31}
I20260806 18:34:44.491276 28394 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:34:44.483955 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.194.62:33607 (local address 127.19.194.2:57257)
0806 18:34:44.484257 (+ 302us) negotiation.cc:107] Waiting for socket to connect
0806 18:34:44.484274 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0806 18:34:44.484382 (+ 108us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:34:44.484640 (+ 258us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:34:44.484645 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:34:44.484855 (+ 210us) client_negotiation.cc:189] Negotiated authn=SASL
0806 18:34:44.484952 (+ 97us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:34:44.484957 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:34:44.485766 (+ 809us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:34:44.485769 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:34:44.486335 (+ 566us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:34:44.486342 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:34:44.486919 (+ 577us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:34:44.486923 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:34:44.487015 (+ 92us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:34:44.487459 (+ 444us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 18:34:44.487480 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0806 18:34:44.487847 (+ 367us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 18:34:44.489486 (+ 1639us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:34:44.489492 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:34:44.489495 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:34:44.489772 (+ 277us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:34:44.490041 (+ 269us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:34:44.490047 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:34:44.490052 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:34:44.490157 (+ 105us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:34:44.490553 (+ 396us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 18:34:44.490559 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 18:34:44.490678 (+ 119us) client_negotiation.cc:769] Sending connection context
0806 18:34:44.490808 (+ 130us) client_negotiation.cc:240] Negotiation successful
0806 18:34:44.490969 (+ 161us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":196,"thread_start_us":90,"threads_started":1}
WARNING: no policy specified for oryx/127.19.194.3@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.19.194.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.19.194.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786041246094603-20232-0/krb5kdc/oryx_127.19.194.3.keytab.
Entry for principal oryx/127.19.194.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786041246094603-20232-0/krb5kdc/oryx_127.19.194.3.keytab.
I20260806 18:34:44.512235 20232 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJW69tn/build/debug/bin/kudu
/tmp/dist-test-taskJW69tn/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786041246094603-20232-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786041246094603-20232-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786041246094603-20232-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786041246094603-20232-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.19.194.3:0
--local_ip_for_outbound_sockets=127.19.194.3
--webserver_interface=127.19.194.3
--webserver_port=0
--tserver_master_addrs=127.19.194.62:33607
--keytab_file=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786041246094603-20232-0/krb5kdc/oryx_127.19.194.3.keytab
--principal=oryx/127.19.194.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.19.194.20:36279
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786041246094603-20232-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786041246094603-20232-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786041246094603-20232-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 18:34:44.623982 28397 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 18:34:44.624210 28397 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 18:34:44.624253 28397 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 18:34:44.628158 28397 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 18:34:44.628207 28397 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 18:34:44.628244 28397 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260806 18:34:44.628276 28397 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.19.194.3
I20260806 18:34:44.633371 28397 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.194.20:36279
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786041246094603-20232-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786041246094603-20232-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786041246094603-20232-0/krb5kdc/oryx_127.19.194.3.keytab
--principal=oryx/127.19.194.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.19.194.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786041246094603-20232-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.19.194.3
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.19.194.62:33607
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28397
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.19.194.3
--log_dir=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786041246094603-20232-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 17:59:13 UTC on ccycloud.quasar-wvsnzx.root.comops.site
build id 4571028
I20260806 18:34:44.634258 28397 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 18:34:44.635231 28397 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 18:34:44.644692 28403 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:34:44.644692 28402 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:34:44.644769 28405 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 18:34:44.644804 28397 server_base.cc:1034] running on GCE node
I20260806 18:34:44.645294 28397 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 18:34:44.645980 28397 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 18:34:44.647194 28397 hybrid_clock.cc:648] HybridClock initialized: now 1786041284647158 us; error 45 us; skew 500 ppm
Aug 06 18:34:44 dist-test-slave-1xrh krb5kdc[27989](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041284, etypes {rep=17 tkt=17 ses=17}, oryx/127.19.194.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 18:34:44.649926 28397 init.cc:377] Logged in from keytab as oryx/127.19.194.3@KRBTEST.COM (short username oryx)
I20260806 18:34:44.650729 28397 webserver.cc:460] Webserver started at http://127.19.194.3:46177/ using document root <none> and password file <none>
I20260806 18:34:44.651342 28397 fs_manager.cc:359] Metadata directory not provided
I20260806 18:34:44.651433 28397 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 18:34:44.651659 28397 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 18:34:44.653590 28397 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786041246094603-20232-0/minicluster-data/ts-2/data/instance:
uuid: "63955300c20044878851d5e44e44155c"
format_stamp: "Formatted at 2026-08-06 18:34:44 on dist-test-slave-1xrh"
server_key: "aac2d444095419599a9f5230a5d5bf0f"
server_key_iv: "2c6e1547f967dac9e95de7d23f63fb7c"
server_key_version: "encryptionkey@0"
I20260806 18:34:44.654155 28397 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786041246094603-20232-0/minicluster-data/ts-2/wal/instance:
uuid: "63955300c20044878851d5e44e44155c"
format_stamp: "Formatted at 2026-08-06 18:34:44 on dist-test-slave-1xrh"
server_key: "aac2d444095419599a9f5230a5d5bf0f"
server_key_iv: "2c6e1547f967dac9e95de7d23f63fb7c"
server_key_version: "encryptionkey@0"
I20260806 18:34:44.657792 28397 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.006s sys 0.000s
I20260806 18:34:44.660032 28412 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 18:34:44.661129 28397 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260806 18:34:44.661266 28397 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786041246094603-20232-0/minicluster-data/ts-2/data,/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786041246094603-20232-0/minicluster-data/ts-2/wal
uuid: "63955300c20044878851d5e44e44155c"
format_stamp: "Formatted at 2026-08-06 18:34:44 on dist-test-slave-1xrh"
server_key: "aac2d444095419599a9f5230a5d5bf0f"
server_key_iv: "2c6e1547f967dac9e95de7d23f63fb7c"
server_key_version: "encryptionkey@0"
I20260806 18:34:44.661418 28397 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786041246094603-20232-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786041246094603-20232-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786041246094603-20232-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 18:34:44.676860 28397 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 18:34:44.679574 28397 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 18:34:44.679761 28397 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 18:34:44.681321 28397 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 18:34:44.681380 28397 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 18:34:44.681428 28397 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 18:34:44.681459 28397 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 06 18:34:44 dist-test-slave-1xrh krb5kdc[27989](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041284, etypes {rep=17 tkt=17 ses=17}, oryx/127.19.194.3@KRBTEST.COM for oryx/127.19.194.62@KRBTEST.COM
I20260806 18:34:44.692616 28397 rpc_server.cc:307] RPC server started. Bound to: 127.19.194.3:46307
I20260806 18:34:44.692773 28530 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.194.3:46307 every 8 connection(s)
I20260806 18:34:44.693650 28397 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786041246094603-20232-0/minicluster-data/ts-2/data/info.pb
I20260806 18:34:44.695405 28085 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:34:44.682164 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.194.3:55741 (local address 127.19.194.62:33607)
0806 18:34:44.682328 (+ 164us) server_negotiation.cc:206] Beginning negotiation
0806 18:34:44.682334 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 18:34:44.683263 (+ 929us) server_negotiation.cc:407] Connection header received
0806 18:34:44.684323 (+ 1060us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:34:44.684327 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:34:44.684390 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:34:44.684481 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:34:44.686183 (+ 1702us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:34:44.687040 (+ 857us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:34:44.688408 (+ 1368us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:34:44.688707 (+ 299us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:34:44.688832 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:34:44.691464 (+ 2632us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:34:44.691472 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:34:44.691475 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 18:34:44.691541 (+ 66us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:34:44.693218 (+ 1677us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:34:44.693752 (+ 534us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:34:44.693756 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:34:44.693758 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:34:44.693851 (+ 93us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:34:44.694239 (+ 388us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:34:44.694242 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:34:44.694244 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:34:44.694481 (+ 237us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:34:44.694583 (+ 102us) server_negotiation.cc:1035] Waiting for connection context
0806 18:34:44.695099 (+ 516us) server_negotiation.cc:299] Negotiation successful
0806 18:34:44.695260 (+ 161us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260806 18:34:44.696159 28421 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:34:44.682624 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.194.62:33607 (local address 127.19.194.3:55741)
0806 18:34:44.683091 (+ 467us) negotiation.cc:107] Waiting for socket to connect
0806 18:34:44.683159 (+ 68us) client_negotiation.cc:174] Beginning negotiation
0806 18:34:44.684018 (+ 859us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:34:44.684924 (+ 906us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:34:44.684933 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:34:44.685414 (+ 481us) client_negotiation.cc:189] Negotiated authn=SASL
0806 18:34:44.686033 (+ 619us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:34:44.686057 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:34:44.687194 (+ 1137us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:34:44.687197 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:34:44.688262 (+ 1065us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:34:44.688273 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:34:44.688886 (+ 613us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:34:44.688890 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:34:44.689066 (+ 176us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:34:44.689617 (+ 551us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 18:34:44.689633 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0806 18:34:44.691296 (+ 1663us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 18:34:44.693333 (+ 2037us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:34:44.693338 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:34:44.693349 (+ 11us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:34:44.693597 (+ 248us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:34:44.693954 (+ 357us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:34:44.693956 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:34:44.693958 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:34:44.694095 (+ 137us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:34:44.694643 (+ 548us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 18:34:44.694648 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 18:34:44.694848 (+ 200us) client_negotiation.cc:769] Sending connection context
0806 18:34:44.694993 (+ 145us) client_negotiation.cc:240] Negotiation successful
0806 18:34:44.695326 (+ 333us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":263,"thread_start_us":143,"threads_started":1}
I20260806 18:34:44.697515 28531 heartbeater.cc:347] Connected to a master server at 127.19.194.62:33607
I20260806 18:34:44.697710 28531 heartbeater.cc:464] Registering TS with master...
I20260806 18:34:44.698235 28531 heartbeater.cc:511] Master 127.19.194.62:33607 requested a full tablet report, sending...
I20260806 18:34:44.698408 20232 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJW69tn/build/debug/bin/kudu as pid 28397
I20260806 18:34:44.698541 20232 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786041246094603-20232-0/minicluster-data/ts-2/wal/instance
I20260806 18:34:44.698876 20232 external_mini_cluster.cc:1428] Setting key 80e8fe6e237e3373b0b5781a8fff9525
I20260806 18:34:44.699389 28034 ts_manager.cc:194] Registered new tserver with Master: 63955300c20044878851d5e44e44155c (127.19.194.3:46307)
I20260806 18:34:44.700129 28034 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.19.194.3@KRBTEST.COM'} at 127.19.194.3:55741
I20260806 18:34:44.701822 20232 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260806 18:34:44.707285 28085 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:34:44.699799 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.194.3:48199 (local address 127.19.194.62:33607)
0806 18:34:44.699930 (+ 131us) server_negotiation.cc:206] Beginning negotiation
0806 18:34:44.699934 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 18:34:44.700106 (+ 172us) server_negotiation.cc:407] Connection header received
0806 18:34:44.700233 (+ 127us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:34:44.700236 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:34:44.700289 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:34:44.700352 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:34:44.700891 (+ 539us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:34:44.701577 (+ 686us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:34:44.702508 (+ 931us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:34:44.702881 (+ 373us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:34:44.703020 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:34:44.704213 (+ 1193us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:34:44.704216 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:34:44.704217 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 18:34:44.704238 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:34:44.705550 (+ 1312us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:34:44.706064 (+ 514us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:34:44.706066 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:34:44.706067 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:34:44.706136 (+ 69us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:34:44.706441 (+ 305us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:34:44.706443 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:34:44.706444 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:34:44.706593 (+ 149us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:34:44.706668 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0806 18:34:44.707014 (+ 346us) server_negotiation.cc:299] Negotiation successful
0806 18:34:44.707102 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260806 18:34:44.707394 28537 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:34:44.699722 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.194.62:33607 (local address 127.19.194.3:48199)
0806 18:34:44.700016 (+ 294us) negotiation.cc:107] Waiting for socket to connect
0806 18:34:44.700031 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0806 18:34:44.700128 (+ 97us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:34:44.700383 (+ 255us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:34:44.700387 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:34:44.700625 (+ 238us) client_negotiation.cc:189] Negotiated authn=SASL
0806 18:34:44.700748 (+ 123us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:34:44.700756 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:34:44.701734 (+ 978us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:34:44.701739 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:34:44.702384 (+ 645us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:34:44.702393 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:34:44.703013 (+ 620us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:34:44.703015 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:34:44.703096 (+ 81us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:34:44.703642 (+ 546us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 18:34:44.703656 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0806 18:34:44.704072 (+ 416us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 18:34:44.705667 (+ 1595us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:34:44.705673 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:34:44.705678 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:34:44.705948 (+ 270us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:34:44.706231 (+ 283us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:34:44.706237 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:34:44.706242 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:34:44.706347 (+ 105us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:34:44.706699 (+ 352us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 18:34:44.706705 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 18:34:44.706820 (+ 115us) client_negotiation.cc:769] Sending connection context
0806 18:34:44.706949 (+ 129us) client_negotiation.cc:240] Negotiation successful
0806 18:34:44.707164 (+ 215us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":211,"thread_start_us":125,"threads_started":1}
Aug 06 18:34:44 dist-test-slave-1xrh krb5kdc[27989](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041284, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 06 18:34:44 dist-test-slave-1xrh krb5kdc[27989](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-user@KRBTEST.COM for kudu/127.19.194.62@KRBTEST.COM, Server not found in Kerberos database
Aug 06 18:34:44 dist-test-slave-1xrh krb5kdc[27989](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-user@KRBTEST.COM for kudu/127.19.194.62@KRBTEST.COM, Server not found in Kerberos database
W20260806 18:34:44.726403 28085 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 18:34:44.720271 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46774 (local address 127.19.194.62:33607)
0806 18:34:44.720400 (+ 129us) server_negotiation.cc:206] Beginning negotiation
0806 18:34:44.720403 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 18:34:44.720414 (+ 11us) server_negotiation.cc:407] Connection header received
0806 18:34:44.720443 (+ 29us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:34:44.720445 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:34:44.720479 (+ 34us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:34:44.720565 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:34:44.721270 (+ 705us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:34:44.721811 (+ 541us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:34:44.722722 (+ 911us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:34:44.722993 (+ 271us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:34:44.723098 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:34:44.726300 (+ 3202us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:46774: BlockingRecv error: recv got EOF from 127.0.0.1:46774 (error 108)
Metrics: {"server-negotiator.queue_time_us":54}
Aug 06 18:34:44 dist-test-slave-1xrh krb5kdc[27989](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041284, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for oryx/127.19.194.62@KRBTEST.COM
I20260806 18:34:44.735788 28085 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:34:44.728054 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46786 (local address 127.19.194.62:33607)
0806 18:34:44.728192 (+ 138us) server_negotiation.cc:206] Beginning negotiation
0806 18:34:44.728196 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 18:34:44.728328 (+ 132us) server_negotiation.cc:407] Connection header received
0806 18:34:44.728478 (+ 150us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:34:44.728481 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:34:44.728522 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:34:44.728600 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:34:44.729169 (+ 569us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:34:44.729647 (+ 478us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:34:44.730451 (+ 804us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:34:44.730659 (+ 208us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:34:44.730770 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:34:44.732948 (+ 2178us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:34:44.732952 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:34:44.732959 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 18:34:44.732986 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:34:44.734122 (+ 1136us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:34:44.734536 (+ 414us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:34:44.734544 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:34:44.734546 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:34:44.734618 (+ 72us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:34:44.734919 (+ 301us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:34:44.734923 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:34:44.734925 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:34:44.735097 (+ 172us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:34:44.735287 (+ 190us) server_negotiation.cc:1035] Waiting for connection context
0806 18:34:44.735519 (+ 232us) server_negotiation.cc:299] Negotiation successful
0806 18:34:44.735629 (+ 110us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260806 18:34:44.738622 28035 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:46786:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260806 18:34:44.741765 28035 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 18:34:44.756230 28557 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:34:44.750841 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51842 (local address 127.19.194.2:44765)
0806 18:34:44.751293 (+ 452us) server_negotiation.cc:206] Beginning negotiation
0806 18:34:44.751305 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0806 18:34:44.751328 (+ 23us) server_negotiation.cc:407] Connection header received
0806 18:34:44.751399 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:34:44.751403 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:34:44.751602 (+ 199us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:34:44.751759 (+ 157us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 18:34:44.752070 (+ 311us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:34:44.753162 (+ 1092us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:34:44.754887 (+ 1725us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:34:44.755534 (+ 647us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:34:44.755660 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:34:44.755820 (+ 160us) server_negotiation.cc:1035] Waiting for connection context
0806 18:34:44.755954 (+ 134us) server_negotiation.cc:299] Negotiation successful
0806 18:34:44.756056 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":352,"thread_start_us":111,"threads_started":1}
I20260806 18:34:44.756244 28554 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:34:44.750597 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.194.1:34859 (local address 127.0.0.1:33838)
0806 18:34:44.751344 (+ 747us) negotiation.cc:107] Waiting for socket to connect
0806 18:34:44.751357 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0806 18:34:44.751441 (+ 84us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:34:44.752488 (+ 1047us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:34:44.752491 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:34:44.752510 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 18:34:44.752603 (+ 93us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:34:44.752608 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:34:44.753723 (+ 1115us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:34:44.753727 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:34:44.754762 (+ 1035us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:34:44.754772 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:34:44.755777 (+ 1005us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:34:44.755781 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:34:44.755875 (+ 94us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:34:44.755902 (+ 27us) client_negotiation.cc:769] Sending connection context
0806 18:34:44.755952 (+ 50us) client_negotiation.cc:240] Negotiation successful
0806 18:34:44.756061 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":689,"mutex_wait_us":39,"thread_start_us":93,"threads_started":1}
I20260806 18:34:44.756247 28556 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:34:44.750328 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.194.2:44765 (local address 127.0.0.1:51842)
0806 18:34:44.750806 (+ 478us) negotiation.cc:107] Waiting for socket to connect
0806 18:34:44.750851 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0806 18:34:44.750979 (+ 128us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:34:44.751767 (+ 788us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:34:44.751773 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:34:44.751794 (+ 21us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 18:34:44.751920 (+ 126us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:34:44.751929 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:34:44.753291 (+ 1362us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:34:44.753294 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:34:44.754734 (+ 1440us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:34:44.754749 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:34:44.755664 (+ 915us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:34:44.755668 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:34:44.755770 (+ 102us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:34:44.755811 (+ 41us) client_negotiation.cc:769] Sending connection context
0806 18:34:44.755923 (+ 112us) client_negotiation.cc:240] Negotiation successful
0806 18:34:44.756074 (+ 151us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":417,"mutex_wait_us":130,"spinlock_wait_cycles":15744,"thread_start_us":94,"threads_started":1}
I20260806 18:34:44.756776 28553 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:34:44.750133 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.194.3:46307 (local address 127.0.0.1:41986)
0806 18:34:44.750796 (+ 663us) negotiation.cc:107] Waiting for socket to connect
0806 18:34:44.750843 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0806 18:34:44.750979 (+ 136us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:34:44.751917 (+ 938us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:34:44.751922 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:34:44.751939 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 18:34:44.752042 (+ 103us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:34:44.752047 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:34:44.753103 (+ 1056us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:34:44.753105 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:34:44.754768 (+ 1663us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:34:44.754776 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:34:44.756375 (+ 1599us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:34:44.756379 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:34:44.756473 (+ 94us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:34:44.756504 (+ 31us) client_negotiation.cc:769] Sending connection context
0806 18:34:44.756580 (+ 76us) client_negotiation.cc:240] Negotiation successful
0806 18:34:44.756644 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":506,"mutex_wait_us":209,"spinlock_wait_cycles":2816,"thread_start_us":70,"threads_started":1}
I20260806 18:34:44.757073 28558 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:34:44.751023 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33838 (local address 127.19.194.1:34859)
0806 18:34:44.752055 (+ 1032us) server_negotiation.cc:206] Beginning negotiation
0806 18:34:44.752070 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0806 18:34:44.752092 (+ 22us) server_negotiation.cc:407] Connection header received
0806 18:34:44.752161 (+ 69us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:34:44.752165 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:34:44.752336 (+ 171us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:34:44.752479 (+ 143us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 18:34:44.752734 (+ 255us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:34:44.753614 (+ 880us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:34:44.754884 (+ 1270us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:34:44.755637 (+ 753us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:34:44.756617 (+ 980us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:34:44.756767 (+ 150us) server_negotiation.cc:1035] Waiting for connection context
0806 18:34:44.756845 (+ 78us) server_negotiation.cc:299] Negotiation successful
0806 18:34:44.756941 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":926,"thread_start_us":105,"threads_started":1}
I20260806 18:34:44.757201 28555 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:34:44.750310 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41986 (local address 127.19.194.3:46307)
0806 18:34:44.751437 (+ 1127us) server_negotiation.cc:206] Beginning negotiation
0806 18:34:44.751447 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0806 18:34:44.751464 (+ 17us) server_negotiation.cc:407] Connection header received
0806 18:34:44.751527 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:34:44.751531 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:34:44.751663 (+ 132us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:34:44.751785 (+ 122us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 18:34:44.752157 (+ 372us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:34:44.752991 (+ 834us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:34:44.755263 (+ 2272us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:34:44.756016 (+ 753us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:34:44.756133 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:34:44.756305 (+ 172us) server_negotiation.cc:1035] Waiting for connection context
0806 18:34:44.756949 (+ 644us) server_negotiation.cc:299] Negotiation successful
0806 18:34:44.757046 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1038,"thread_start_us":123,"threads_started":1}
I20260806 18:34:44.759475 28313 tablet_service.cc:1467] Processing CreateTablet for tablet eff105295272444ca25e169aa77c29fc (DEFAULT_TABLE table=test-table [id=117ef29ae96743be9deb026043e7d921]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 18:34:44.759505 28462 tablet_service.cc:1467] Processing CreateTablet for tablet eff105295272444ca25e169aa77c29fc (DEFAULT_TABLE table=test-table [id=117ef29ae96743be9deb026043e7d921]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 18:34:44.760550 28313 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet eff105295272444ca25e169aa77c29fc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 18:34:44.760550 28462 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet eff105295272444ca25e169aa77c29fc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 18:34:44.765230 28173 tablet_service.cc:1467] Processing CreateTablet for tablet eff105295272444ca25e169aa77c29fc (DEFAULT_TABLE table=test-table [id=117ef29ae96743be9deb026043e7d921]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 18:34:44.766245 28173 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet eff105295272444ca25e169aa77c29fc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 18:34:44.768678 28559 tablet_bootstrap.cc:492] T eff105295272444ca25e169aa77c29fc P 4575246688ab47bdb5abf0da5c702767: Bootstrap starting.
I20260806 18:34:44.769101 28560 tablet_bootstrap.cc:492] T eff105295272444ca25e169aa77c29fc P 63955300c20044878851d5e44e44155c: Bootstrap starting.
I20260806 18:34:44.771471 28560 tablet_bootstrap.cc:654] T eff105295272444ca25e169aa77c29fc P 63955300c20044878851d5e44e44155c: Neither blocks nor log segments found. Creating new log.
I20260806 18:34:44.772462 28560 log.cc:824] T eff105295272444ca25e169aa77c29fc P 63955300c20044878851d5e44e44155c: Log is configured to *not* fsync() on all Append() calls
I20260806 18:34:44.773041 28559 tablet_bootstrap.cc:654] T eff105295272444ca25e169aa77c29fc P 4575246688ab47bdb5abf0da5c702767: Neither blocks nor log segments found. Creating new log.
I20260806 18:34:44.773823 28559 log.cc:824] T eff105295272444ca25e169aa77c29fc P 4575246688ab47bdb5abf0da5c702767: Log is configured to *not* fsync() on all Append() calls
I20260806 18:34:44.774232 28562 tablet_bootstrap.cc:492] T eff105295272444ca25e169aa77c29fc P 885703862b27424890515f701c83aa68: Bootstrap starting.
I20260806 18:34:44.774819 28560 tablet_bootstrap.cc:492] T eff105295272444ca25e169aa77c29fc P 63955300c20044878851d5e44e44155c: No bootstrap required, opened a new log
I20260806 18:34:44.775012 28560 ts_tablet_manager.cc:1397] T eff105295272444ca25e169aa77c29fc P 63955300c20044878851d5e44e44155c: Time spent bootstrapping tablet: real 0.006s user 0.005s sys 0.000s
I20260806 18:34:44.776171 28562 tablet_bootstrap.cc:654] T eff105295272444ca25e169aa77c29fc P 885703862b27424890515f701c83aa68: Neither blocks nor log segments found. Creating new log.
I20260806 18:34:44.776798 28559 tablet_bootstrap.cc:492] T eff105295272444ca25e169aa77c29fc P 4575246688ab47bdb5abf0da5c702767: No bootstrap required, opened a new log
I20260806 18:34:44.776988 28559 ts_tablet_manager.cc:1397] T eff105295272444ca25e169aa77c29fc P 4575246688ab47bdb5abf0da5c702767: Time spent bootstrapping tablet: real 0.008s user 0.003s sys 0.000s
I20260806 18:34:44.777343 28562 log.cc:824] T eff105295272444ca25e169aa77c29fc P 885703862b27424890515f701c83aa68: Log is configured to *not* fsync() on all Append() calls
I20260806 18:34:44.778782 28562 tablet_bootstrap.cc:492] T eff105295272444ca25e169aa77c29fc P 885703862b27424890515f701c83aa68: No bootstrap required, opened a new log
I20260806 18:34:44.778981 28562 ts_tablet_manager.cc:1397] T eff105295272444ca25e169aa77c29fc P 885703862b27424890515f701c83aa68: Time spent bootstrapping tablet: real 0.005s user 0.004s sys 0.000s
I20260806 18:34:44.778947 28560 raft_consensus.cc:348] T eff105295272444ca25e169aa77c29fc P 63955300c20044878851d5e44e44155c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "63955300c20044878851d5e44e44155c" member_type: VOTER last_known_addr { host: "127.19.194.3" port: 46307 } } peers { permanent_uuid: "4575246688ab47bdb5abf0da5c702767" member_type: VOTER last_known_addr { host: "127.19.194.2" port: 44765 } } peers { permanent_uuid: "885703862b27424890515f701c83aa68" member_type: VOTER last_known_addr { host: "127.19.194.1" port: 34859 } }
I20260806 18:34:44.779424 28560 raft_consensus.cc:374] T eff105295272444ca25e169aa77c29fc P 63955300c20044878851d5e44e44155c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 18:34:44.779534 28560 raft_consensus.cc:729] T eff105295272444ca25e169aa77c29fc P 63955300c20044878851d5e44e44155c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 63955300c20044878851d5e44e44155c, State: Initialized, Role: FOLLOWER
I20260806 18:34:44.780010 28560 consensus_queue.cc:260] T eff105295272444ca25e169aa77c29fc P 63955300c20044878851d5e44e44155c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "63955300c20044878851d5e44e44155c" member_type: VOTER last_known_addr { host: "127.19.194.3" port: 46307 } } peers { permanent_uuid: "4575246688ab47bdb5abf0da5c702767" member_type: VOTER last_known_addr { host: "127.19.194.2" port: 44765 } } peers { permanent_uuid: "885703862b27424890515f701c83aa68" member_type: VOTER last_known_addr { host: "127.19.194.1" port: 34859 } }
I20260806 18:34:44.780681 28531 heartbeater.cc:503] Master 127.19.194.62:33607 was elected leader, sending a full tablet report...
I20260806 18:34:44.781064 28560 ts_tablet_manager.cc:1428] T eff105295272444ca25e169aa77c29fc P 63955300c20044878851d5e44e44155c: Time spent starting tablet: real 0.006s user 0.006s sys 0.000s
I20260806 18:34:44.780968 28559 raft_consensus.cc:348] T eff105295272444ca25e169aa77c29fc P 4575246688ab47bdb5abf0da5c702767 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "63955300c20044878851d5e44e44155c" member_type: VOTER last_known_addr { host: "127.19.194.3" port: 46307 } } peers { permanent_uuid: "4575246688ab47bdb5abf0da5c702767" member_type: VOTER last_known_addr { host: "127.19.194.2" port: 44765 } } peers { permanent_uuid: "885703862b27424890515f701c83aa68" member_type: VOTER last_known_addr { host: "127.19.194.1" port: 34859 } }
I20260806 18:34:44.781262 28559 raft_consensus.cc:374] T eff105295272444ca25e169aa77c29fc P 4575246688ab47bdb5abf0da5c702767 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 18:34:44.781322 28559 raft_consensus.cc:729] T eff105295272444ca25e169aa77c29fc P 4575246688ab47bdb5abf0da5c702767 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4575246688ab47bdb5abf0da5c702767, State: Initialized, Role: FOLLOWER
I20260806 18:34:44.781736 28559 consensus_queue.cc:260] T eff105295272444ca25e169aa77c29fc P 4575246688ab47bdb5abf0da5c702767 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "63955300c20044878851d5e44e44155c" member_type: VOTER last_known_addr { host: "127.19.194.3" port: 46307 } } peers { permanent_uuid: "4575246688ab47bdb5abf0da5c702767" member_type: VOTER last_known_addr { host: "127.19.194.2" port: 44765 } } peers { permanent_uuid: "885703862b27424890515f701c83aa68" member_type: VOTER last_known_addr { host: "127.19.194.1" port: 34859 } }
I20260806 18:34:44.782752 28559 ts_tablet_manager.cc:1428] T eff105295272444ca25e169aa77c29fc P 4575246688ab47bdb5abf0da5c702767: Time spent starting tablet: real 0.006s user 0.006s sys 0.000s
I20260806 18:34:44.783111 28388 heartbeater.cc:503] Master 127.19.194.62:33607 was elected leader, sending a full tablet report...
I20260806 18:34:44.784341 28562 raft_consensus.cc:348] T eff105295272444ca25e169aa77c29fc P 885703862b27424890515f701c83aa68 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "63955300c20044878851d5e44e44155c" member_type: VOTER last_known_addr { host: "127.19.194.3" port: 46307 } } peers { permanent_uuid: "4575246688ab47bdb5abf0da5c702767" member_type: VOTER last_known_addr { host: "127.19.194.2" port: 44765 } } peers { permanent_uuid: "885703862b27424890515f701c83aa68" member_type: VOTER last_known_addr { host: "127.19.194.1" port: 34859 } }
I20260806 18:34:44.784590 28562 raft_consensus.cc:374] T eff105295272444ca25e169aa77c29fc P 885703862b27424890515f701c83aa68 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 18:34:44.784651 28562 raft_consensus.cc:729] T eff105295272444ca25e169aa77c29fc P 885703862b27424890515f701c83aa68 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 885703862b27424890515f701c83aa68, State: Initialized, Role: FOLLOWER
I20260806 18:34:44.785087 28562 consensus_queue.cc:260] T eff105295272444ca25e169aa77c29fc P 885703862b27424890515f701c83aa68 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "63955300c20044878851d5e44e44155c" member_type: VOTER last_known_addr { host: "127.19.194.3" port: 46307 } } peers { permanent_uuid: "4575246688ab47bdb5abf0da5c702767" member_type: VOTER last_known_addr { host: "127.19.194.2" port: 44765 } } peers { permanent_uuid: "885703862b27424890515f701c83aa68" member_type: VOTER last_known_addr { host: "127.19.194.1" port: 34859 } }
I20260806 18:34:44.785738 28239 heartbeater.cc:503] Master 127.19.194.62:33607 was elected leader, sending a full tablet report...
I20260806 18:34:44.786759 28562 ts_tablet_manager.cc:1428] T eff105295272444ca25e169aa77c29fc P 885703862b27424890515f701c83aa68: Time spent starting tablet: real 0.008s user 0.003s sys 0.003s
I20260806 18:34:44.855584 28567 raft_consensus.cc:482] T eff105295272444ca25e169aa77c29fc P 885703862b27424890515f701c83aa68 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 18:34:44.855767 28567 raft_consensus.cc:504] T eff105295272444ca25e169aa77c29fc P 885703862b27424890515f701c83aa68 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "63955300c20044878851d5e44e44155c" member_type: VOTER last_known_addr { host: "127.19.194.3" port: 46307 } } peers { permanent_uuid: "4575246688ab47bdb5abf0da5c702767" member_type: VOTER last_known_addr { host: "127.19.194.2" port: 44765 } } peers { permanent_uuid: "885703862b27424890515f701c83aa68" member_type: VOTER last_known_addr { host: "127.19.194.1" port: 34859 } }
I20260806 18:34:44.856921 28567 leader_election.cc:288] T eff105295272444ca25e169aa77c29fc P 885703862b27424890515f701c83aa68 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 63955300c20044878851d5e44e44155c (127.19.194.3:46307), 4575246688ab47bdb5abf0da5c702767 (127.19.194.2:44765)
I20260806 18:34:44.860949 28557 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:34:44.857401 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.194.1:37507 (local address 127.19.194.2:44765)
0806 18:34:44.857560 (+ 159us) server_negotiation.cc:206] Beginning negotiation
0806 18:34:44.857566 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 18:34:44.857863 (+ 297us) server_negotiation.cc:407] Connection header received
0806 18:34:44.858014 (+ 151us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:34:44.858016 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:34:44.858068 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:34:44.858150 (+ 82us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 18:34:44.858416 (+ 266us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:34:44.858903 (+ 487us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:34:44.860029 (+ 1126us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:34:44.860390 (+ 361us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:34:44.860522 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:34:44.860567 (+ 45us) server_negotiation.cc:1035] Waiting for connection context
0806 18:34:44.860761 (+ 194us) server_negotiation.cc:299] Negotiation successful
0806 18:34:44.860829 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260806 18:34:44.860975 28555 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:34:44.857302 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.194.1:49425 (local address 127.19.194.3:46307)
0806 18:34:44.857463 (+ 161us) server_negotiation.cc:206] Beginning negotiation
0806 18:34:44.857467 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 18:34:44.857730 (+ 263us) server_negotiation.cc:407] Connection header received
0806 18:34:44.857890 (+ 160us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:34:44.857893 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:34:44.857945 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:34:44.858034 (+ 89us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 18:34:44.858315 (+ 281us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:34:44.858825 (+ 510us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:34:44.860001 (+ 1176us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:34:44.860376 (+ 375us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:34:44.860504 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:34:44.860565 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0806 18:34:44.860750 (+ 185us) server_negotiation.cc:299] Negotiation successful
0806 18:34:44.860830 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260806 18:34:44.860998 28568 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:34:44.857168 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.194.2:44765 (local address 127.19.194.1:37507)
0806 18:34:44.857763 (+ 595us) negotiation.cc:107] Waiting for socket to connect
0806 18:34:44.857775 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0806 18:34:44.857897 (+ 122us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:34:44.858168 (+ 271us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:34:44.858170 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:34:44.858187 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 18:34:44.858307 (+ 120us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:34:44.858312 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:34:44.859026 (+ 714us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:34:44.859031 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:34:44.859885 (+ 854us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:34:44.859894 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:34:44.860518 (+ 624us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:34:44.860521 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:34:44.860613 (+ 92us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:34:44.860634 (+ 21us) client_negotiation.cc:769] Sending connection context
0806 18:34:44.860733 (+ 99us) client_negotiation.cc:240] Negotiation successful
0806 18:34:44.860821 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":528,"mutex_wait_us":85,"thread_start_us":127,"threads_started":1}
I20260806 18:34:44.861002 28569 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:34:44.857168 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.194.3:46307 (local address 127.19.194.1:49425)
0806 18:34:44.857634 (+ 466us) negotiation.cc:107] Waiting for socket to connect
0806 18:34:44.857647 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0806 18:34:44.857778 (+ 131us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:34:44.858054 (+ 276us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:34:44.858057 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:34:44.858072 (+ 15us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 18:34:44.858195 (+ 123us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:34:44.858200 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:34:44.858950 (+ 750us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:34:44.858954 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:34:44.859874 (+ 920us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:34:44.859883 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:34:44.860498 (+ 615us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:34:44.860502 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:34:44.860601 (+ 99us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:34:44.860621 (+ 20us) client_negotiation.cc:769] Sending connection context
0806 18:34:44.860734 (+ 113us) client_negotiation.cc:240] Negotiation successful
0806 18:34:44.860852 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":409,"mutex_wait_us":54,"thread_start_us":168,"threads_started":1}
I20260806 18:34:44.861682 28334 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "eff105295272444ca25e169aa77c29fc" candidate_uuid: "885703862b27424890515f701c83aa68" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4575246688ab47bdb5abf0da5c702767" is_pre_election: true
I20260806 18:34:44.861759 28465 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "eff105295272444ca25e169aa77c29fc" candidate_uuid: "885703862b27424890515f701c83aa68" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "63955300c20044878851d5e44e44155c" is_pre_election: true
I20260806 18:34:44.861987 28334 raft_consensus.cc:2454] T eff105295272444ca25e169aa77c29fc P 4575246688ab47bdb5abf0da5c702767 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 885703862b27424890515f701c83aa68 in term 0.
I20260806 18:34:44.862080 28465 raft_consensus.cc:2454] T eff105295272444ca25e169aa77c29fc P 63955300c20044878851d5e44e44155c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 885703862b27424890515f701c83aa68 in term 0.
I20260806 18:34:44.862524 28126 leader_election.cc:302] T eff105295272444ca25e169aa77c29fc P 885703862b27424890515f701c83aa68 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 4575246688ab47bdb5abf0da5c702767, 63955300c20044878851d5e44e44155c, 885703862b27424890515f701c83aa68; no voters:
I20260806 18:34:44.862808 28567 raft_consensus.cc:2781] T eff105295272444ca25e169aa77c29fc P 885703862b27424890515f701c83aa68 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 18:34:44.862885 28567 raft_consensus.cc:482] T eff105295272444ca25e169aa77c29fc P 885703862b27424890515f701c83aa68 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 18:34:44.862936 28567 raft_consensus.cc:3037] T eff105295272444ca25e169aa77c29fc P 885703862b27424890515f701c83aa68 [term 0 FOLLOWER]: Advancing to term 1
I20260806 18:34:44.864074 28567 raft_consensus.cc:504] T eff105295272444ca25e169aa77c29fc P 885703862b27424890515f701c83aa68 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "63955300c20044878851d5e44e44155c" member_type: VOTER last_known_addr { host: "127.19.194.3" port: 46307 } } peers { permanent_uuid: "4575246688ab47bdb5abf0da5c702767" member_type: VOTER last_known_addr { host: "127.19.194.2" port: 44765 } } peers { permanent_uuid: "885703862b27424890515f701c83aa68" member_type: VOTER last_known_addr { host: "127.19.194.1" port: 34859 } }
I20260806 18:34:44.864473 28567 leader_election.cc:288] T eff105295272444ca25e169aa77c29fc P 885703862b27424890515f701c83aa68 [CANDIDATE]: Term 1 election: Requested vote from peers 63955300c20044878851d5e44e44155c (127.19.194.3:46307), 4575246688ab47bdb5abf0da5c702767 (127.19.194.2:44765)
I20260806 18:34:44.864758 28465 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "eff105295272444ca25e169aa77c29fc" candidate_uuid: "885703862b27424890515f701c83aa68" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "63955300c20044878851d5e44e44155c"
I20260806 18:34:44.864881 28465 raft_consensus.cc:3037] T eff105295272444ca25e169aa77c29fc P 63955300c20044878851d5e44e44155c [term 0 FOLLOWER]: Advancing to term 1
I20260806 18:34:44.864880 28334 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "eff105295272444ca25e169aa77c29fc" candidate_uuid: "885703862b27424890515f701c83aa68" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4575246688ab47bdb5abf0da5c702767"
I20260806 18:34:44.865029 28334 raft_consensus.cc:3037] T eff105295272444ca25e169aa77c29fc P 4575246688ab47bdb5abf0da5c702767 [term 0 FOLLOWER]: Advancing to term 1
I20260806 18:34:44.866088 28465 raft_consensus.cc:2454] T eff105295272444ca25e169aa77c29fc P 63955300c20044878851d5e44e44155c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 885703862b27424890515f701c83aa68 in term 1.
I20260806 18:34:44.866132 28334 raft_consensus.cc:2454] T eff105295272444ca25e169aa77c29fc P 4575246688ab47bdb5abf0da5c702767 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 885703862b27424890515f701c83aa68 in term 1.
I20260806 18:34:44.866420 28126 leader_election.cc:302] T eff105295272444ca25e169aa77c29fc P 885703862b27424890515f701c83aa68 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4575246688ab47bdb5abf0da5c702767, 885703862b27424890515f701c83aa68; no voters:
I20260806 18:34:44.866624 28567 raft_consensus.cc:2781] T eff105295272444ca25e169aa77c29fc P 885703862b27424890515f701c83aa68 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 18:34:44.866822 28567 raft_consensus.cc:686] T eff105295272444ca25e169aa77c29fc P 885703862b27424890515f701c83aa68 [term 1 LEADER]: Becoming Leader. State: Replica: 885703862b27424890515f701c83aa68, State: Running, Role: LEADER
I20260806 18:34:44.867172 28567 consensus_queue.cc:237] T eff105295272444ca25e169aa77c29fc P 885703862b27424890515f701c83aa68 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "63955300c20044878851d5e44e44155c" member_type: VOTER last_known_addr { host: "127.19.194.3" port: 46307 } } peers { permanent_uuid: "4575246688ab47bdb5abf0da5c702767" member_type: VOTER last_known_addr { host: "127.19.194.2" port: 44765 } } peers { permanent_uuid: "885703862b27424890515f701c83aa68" member_type: VOTER last_known_addr { host: "127.19.194.1" port: 34859 } }
I20260806 18:34:44.870450 28034 catalog_manager.cc:5495] T eff105295272444ca25e169aa77c29fc P 885703862b27424890515f701c83aa68 reported cstate change: term changed from 0 to 1, leader changed from <none> to 885703862b27424890515f701c83aa68 (127.19.194.1). New cstate: current_term: 1 leader_uuid: "885703862b27424890515f701c83aa68" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "63955300c20044878851d5e44e44155c" member_type: VOTER last_known_addr { host: "127.19.194.3" port: 46307 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4575246688ab47bdb5abf0da5c702767" member_type: VOTER last_known_addr { host: "127.19.194.2" port: 44765 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "885703862b27424890515f701c83aa68" member_type: VOTER last_known_addr { host: "127.19.194.1" port: 34859 } health_report { overall_health: HEALTHY } } }
I20260806 18:34:44.907704 28558 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:34:44.904266 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33842 (local address 127.19.194.1:34859)
0806 18:34:44.904401 (+ 135us) server_negotiation.cc:206] Beginning negotiation
0806 18:34:44.904405 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 18:34:44.904418 (+ 13us) server_negotiation.cc:407] Connection header received
0806 18:34:44.904500 (+ 82us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:34:44.904503 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:34:44.904566 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:34:44.904652 (+ 86us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 18:34:44.904881 (+ 229us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:34:44.905368 (+ 487us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:34:44.906187 (+ 819us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:34:44.906409 (+ 222us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:34:44.906492 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:34:44.906744 (+ 252us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 18:34:44.907184 (+ 440us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 18:34:44.907271 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0806 18:34:44.907356 (+ 85us) server_negotiation.cc:299] Negotiation successful
0806 18:34:44.907409 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260806 18:34:44.913686 28334 raft_consensus.cc:1261] T eff105295272444ca25e169aa77c29fc P 4575246688ab47bdb5abf0da5c702767 [term 1 FOLLOWER]: Refusing update from remote peer 885703862b27424890515f701c83aa68: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 18:34:44.913970 28465 raft_consensus.cc:1261] T eff105295272444ca25e169aa77c29fc P 63955300c20044878851d5e44e44155c [term 1 FOLLOWER]: Refusing update from remote peer 885703862b27424890515f701c83aa68: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 18:34:44.914474 28570 consensus_queue.cc:1035] T eff105295272444ca25e169aa77c29fc P 885703862b27424890515f701c83aa68 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4575246688ab47bdb5abf0da5c702767" member_type: VOTER last_known_addr { host: "127.19.194.2" port: 44765 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 18:34:44.914675 28567 consensus_queue.cc:1035] T eff105295272444ca25e169aa77c29fc P 885703862b27424890515f701c83aa68 [LEADER]: Connected to new peer: Peer: permanent_uuid: "63955300c20044878851d5e44e44155c" member_type: VOTER last_known_addr { host: "127.19.194.3" port: 46307 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 18:34:44.931871 28033 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:46786:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 18:34:44.932044 28033 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:46786:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 18:34:44.934422 28033 catalog_manager.cc:5782] T 00000000000000000000000000000000 P d0282f2efb864823bbf90a21992ab39d: Sending DeleteTablet for 3 replicas of tablet eff105295272444ca25e169aa77c29fc
I20260806 18:34:44.935160 28313 tablet_service.cc:1514] Processing DeleteTablet for tablet eff105295272444ca25e169aa77c29fc with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 18:34:44 UTC) from {username='oryx', principal='oryx/127.19.194.62@KRBTEST.COM'} at 127.0.0.1:51842
I20260806 18:34:44.935160 28462 tablet_service.cc:1514] Processing DeleteTablet for tablet eff105295272444ca25e169aa77c29fc with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 18:34:44 UTC) from {username='oryx', principal='oryx/127.19.194.62@KRBTEST.COM'} at 127.0.0.1:41986
I20260806 18:34:44.935344 28173 tablet_service.cc:1514] Processing DeleteTablet for tablet eff105295272444ca25e169aa77c29fc with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 18:34:44 UTC) from {username='oryx', principal='oryx/127.19.194.62@KRBTEST.COM'} at 127.0.0.1:33838
I20260806 18:34:44.935817 28584 tablet_replica.cc:330] T eff105295272444ca25e169aa77c29fc P 4575246688ab47bdb5abf0da5c702767: stopping tablet replica
I20260806 18:34:44.936142 28585 tablet_replica.cc:330] T eff105295272444ca25e169aa77c29fc P 885703862b27424890515f701c83aa68: stopping tablet replica
I20260806 18:34:44.936141 28584 raft_consensus.cc:2229] T eff105295272444ca25e169aa77c29fc P 4575246688ab47bdb5abf0da5c702767 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 18:34:44.936425 28584 raft_consensus.cc:2258] T eff105295272444ca25e169aa77c29fc P 4575246688ab47bdb5abf0da5c702767 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 18:34:44.936393 28033 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:46786:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260806 18:34:44.936793 28583 tablet_replica.cc:330] T eff105295272444ca25e169aa77c29fc P 63955300c20044878851d5e44e44155c: stopping tablet replica
W20260806 18:34:44.936796 28033 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 18:34:44.936796 28585 raft_consensus.cc:2229] T eff105295272444ca25e169aa77c29fc P 885703862b27424890515f701c83aa68 [term 1 LEADER]: Raft consensus shutting down.
I20260806 18:34:44.937076 28583 raft_consensus.cc:2229] T eff105295272444ca25e169aa77c29fc P 63955300c20044878851d5e44e44155c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 18:34:44.937361 28583 raft_consensus.cc:2258] T eff105295272444ca25e169aa77c29fc P 63955300c20044878851d5e44e44155c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 18:34:44.937361 28585 raft_consensus.cc:2258] T eff105295272444ca25e169aa77c29fc P 885703862b27424890515f701c83aa68 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 18:34:44.938700 28585 ts_tablet_manager.cc:1905] T eff105295272444ca25e169aa77c29fc P 885703862b27424890515f701c83aa68: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 18:34:44.938700 28583 ts_tablet_manager.cc:1905] T eff105295272444ca25e169aa77c29fc P 63955300c20044878851d5e44e44155c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 18:34:44.941496 28585 ts_tablet_manager.cc:1918] T eff105295272444ca25e169aa77c29fc P 885703862b27424890515f701c83aa68: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 18:34:44.941682 28583 ts_tablet_manager.cc:1918] T eff105295272444ca25e169aa77c29fc P 63955300c20044878851d5e44e44155c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 18:34:44.941769 28583 log.cc:1196] T eff105295272444ca25e169aa77c29fc P 63955300c20044878851d5e44e44155c: Deleting WAL directory at /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786041246094603-20232-0/minicluster-data/ts-2/wal/wals/eff105295272444ca25e169aa77c29fc
I20260806 18:34:44.941921 28462 tablet_service.cc:1467] Processing CreateTablet for tablet 7c62cb709efc4f9d865b0945a50b8db2 (DEFAULT_TABLE table=test-table [id=b0a995d8f6204467b57e7d329e46fc68]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 18:34:44.942117 28583 ts_tablet_manager.cc:1939] T eff105295272444ca25e169aa77c29fc P 63955300c20044878851d5e44e44155c: Deleting consensus metadata
I20260806 18:34:44.942163 28462 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7c62cb709efc4f9d865b0945a50b8db2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 18:34:44.942131 28313 tablet_service.cc:1467] Processing CreateTablet for tablet 7c62cb709efc4f9d865b0945a50b8db2 (DEFAULT_TABLE table=test-table [id=b0a995d8f6204467b57e7d329e46fc68]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 18:34:44.942378 28313 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7c62cb709efc4f9d865b0945a50b8db2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 18:34:44.944044 28560 tablet_bootstrap.cc:492] T 7c62cb709efc4f9d865b0945a50b8db2 P 63955300c20044878851d5e44e44155c: Bootstrap starting.
I20260806 18:34:44.944238 28585 log.cc:1196] T eff105295272444ca25e169aa77c29fc P 885703862b27424890515f701c83aa68: Deleting WAL directory at /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786041246094603-20232-0/minicluster-data/ts-0/wal/wals/eff105295272444ca25e169aa77c29fc
I20260806 18:34:44.944669 28585 ts_tablet_manager.cc:1939] T eff105295272444ca25e169aa77c29fc P 885703862b27424890515f701c83aa68: Deleting consensus metadata
I20260806 18:34:44.944747 28559 tablet_bootstrap.cc:492] T 7c62cb709efc4f9d865b0945a50b8db2 P 4575246688ab47bdb5abf0da5c702767: Bootstrap starting.
I20260806 18:34:44.945122 28560 tablet_bootstrap.cc:654] T 7c62cb709efc4f9d865b0945a50b8db2 P 63955300c20044878851d5e44e44155c: Neither blocks nor log segments found. Creating new log.
I20260806 18:34:44.945104 28019 catalog_manager.cc:4841] TS 63955300c20044878851d5e44e44155c (127.19.194.3:46307): tablet eff105295272444ca25e169aa77c29fc (table test-table [id=117ef29ae96743be9deb026043e7d921]) successfully deleted
I20260806 18:34:44.945744 28559 tablet_bootstrap.cc:654] T 7c62cb709efc4f9d865b0945a50b8db2 P 4575246688ab47bdb5abf0da5c702767: Neither blocks nor log segments found. Creating new log.
I20260806 18:34:44.945708 28173 tablet_service.cc:1467] Processing CreateTablet for tablet 7c62cb709efc4f9d865b0945a50b8db2 (DEFAULT_TABLE table=test-table [id=b0a995d8f6204467b57e7d329e46fc68]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 18:34:44.945955 28173 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7c62cb709efc4f9d865b0945a50b8db2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 18:34:44.945979 28022 catalog_manager.cc:4841] TS 885703862b27424890515f701c83aa68 (127.19.194.1:34859): tablet eff105295272444ca25e169aa77c29fc (table test-table [id=117ef29ae96743be9deb026043e7d921]) successfully deleted
I20260806 18:34:44.946312 28560 tablet_bootstrap.cc:492] T 7c62cb709efc4f9d865b0945a50b8db2 P 63955300c20044878851d5e44e44155c: No bootstrap required, opened a new log
I20260806 18:34:44.946403 28560 ts_tablet_manager.cc:1397] T 7c62cb709efc4f9d865b0945a50b8db2 P 63955300c20044878851d5e44e44155c: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260806 18:34:44.946916 28560 raft_consensus.cc:348] T 7c62cb709efc4f9d865b0945a50b8db2 P 63955300c20044878851d5e44e44155c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "885703862b27424890515f701c83aa68" member_type: VOTER last_known_addr { host: "127.19.194.1" port: 34859 } } peers { permanent_uuid: "4575246688ab47bdb5abf0da5c702767" member_type: VOTER last_known_addr { host: "127.19.194.2" port: 44765 } } peers { permanent_uuid: "63955300c20044878851d5e44e44155c" member_type: VOTER last_known_addr { host: "127.19.194.3" port: 46307 } }
I20260806 18:34:44.947036 28560 raft_consensus.cc:374] T 7c62cb709efc4f9d865b0945a50b8db2 P 63955300c20044878851d5e44e44155c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 18:34:44.947052 28559 tablet_bootstrap.cc:492] T 7c62cb709efc4f9d865b0945a50b8db2 P 4575246688ab47bdb5abf0da5c702767: No bootstrap required, opened a new log
I20260806 18:34:44.947077 28560 raft_consensus.cc:729] T 7c62cb709efc4f9d865b0945a50b8db2 P 63955300c20044878851d5e44e44155c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 63955300c20044878851d5e44e44155c, State: Initialized, Role: FOLLOWER
I20260806 18:34:44.947153 28559 ts_tablet_manager.cc:1397] T 7c62cb709efc4f9d865b0945a50b8db2 P 4575246688ab47bdb5abf0da5c702767: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260806 18:34:44.947273 28560 consensus_queue.cc:260] T 7c62cb709efc4f9d865b0945a50b8db2 P 63955300c20044878851d5e44e44155c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "885703862b27424890515f701c83aa68" member_type: VOTER last_known_addr { host: "127.19.194.1" port: 34859 } } peers { permanent_uuid: "4575246688ab47bdb5abf0da5c702767" member_type: VOTER last_known_addr { host: "127.19.194.2" port: 44765 } } peers { permanent_uuid: "63955300c20044878851d5e44e44155c" member_type: VOTER last_known_addr { host: "127.19.194.3" port: 46307 } }
I20260806 18:34:44.947533 28560 ts_tablet_manager.cc:1428] T 7c62cb709efc4f9d865b0945a50b8db2 P 63955300c20044878851d5e44e44155c: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20260806 18:34:44.947613 28559 raft_consensus.cc:348] T 7c62cb709efc4f9d865b0945a50b8db2 P 4575246688ab47bdb5abf0da5c702767 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "885703862b27424890515f701c83aa68" member_type: VOTER last_known_addr { host: "127.19.194.1" port: 34859 } } peers { permanent_uuid: "4575246688ab47bdb5abf0da5c702767" member_type: VOTER last_known_addr { host: "127.19.194.2" port: 44765 } } peers { permanent_uuid: "63955300c20044878851d5e44e44155c" member_type: VOTER last_known_addr { host: "127.19.194.3" port: 46307 } }
I20260806 18:34:44.947719 28559 raft_consensus.cc:374] T 7c62cb709efc4f9d865b0945a50b8db2 P 4575246688ab47bdb5abf0da5c702767 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 18:34:44.947747 28559 raft_consensus.cc:729] T 7c62cb709efc4f9d865b0945a50b8db2 P 4575246688ab47bdb5abf0da5c702767 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4575246688ab47bdb5abf0da5c702767, State: Initialized, Role: FOLLOWER
I20260806 18:34:44.947840 28559 consensus_queue.cc:260] T 7c62cb709efc4f9d865b0945a50b8db2 P 4575246688ab47bdb5abf0da5c702767 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "885703862b27424890515f701c83aa68" member_type: VOTER last_known_addr { host: "127.19.194.1" port: 34859 } } peers { permanent_uuid: "4575246688ab47bdb5abf0da5c702767" member_type: VOTER last_known_addr { host: "127.19.194.2" port: 44765 } } peers { permanent_uuid: "63955300c20044878851d5e44e44155c" member_type: VOTER last_known_addr { host: "127.19.194.3" port: 46307 } }
I20260806 18:34:44.948082 28559 ts_tablet_manager.cc:1428] T 7c62cb709efc4f9d865b0945a50b8db2 P 4575246688ab47bdb5abf0da5c702767: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20260806 18:34:44.948295 28562 tablet_bootstrap.cc:492] T 7c62cb709efc4f9d865b0945a50b8db2 P 885703862b27424890515f701c83aa68: Bootstrap starting.
I20260806 18:34:44.949141 28562 tablet_bootstrap.cc:654] T 7c62cb709efc4f9d865b0945a50b8db2 P 885703862b27424890515f701c83aa68: Neither blocks nor log segments found. Creating new log.
I20260806 18:34:44.950277 28562 tablet_bootstrap.cc:492] T 7c62cb709efc4f9d865b0945a50b8db2 P 885703862b27424890515f701c83aa68: No bootstrap required, opened a new log
I20260806 18:34:44.950353 28562 ts_tablet_manager.cc:1397] T 7c62cb709efc4f9d865b0945a50b8db2 P 885703862b27424890515f701c83aa68: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.001s
I20260806 18:34:44.950805 28562 raft_consensus.cc:348] T 7c62cb709efc4f9d865b0945a50b8db2 P 885703862b27424890515f701c83aa68 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "885703862b27424890515f701c83aa68" member_type: VOTER last_known_addr { host: "127.19.194.1" port: 34859 } } peers { permanent_uuid: "4575246688ab47bdb5abf0da5c702767" member_type: VOTER last_known_addr { host: "127.19.194.2" port: 44765 } } peers { permanent_uuid: "63955300c20044878851d5e44e44155c" member_type: VOTER last_known_addr { host: "127.19.194.3" port: 46307 } }
I20260806 18:34:44.950910 28562 raft_consensus.cc:374] T 7c62cb709efc4f9d865b0945a50b8db2 P 885703862b27424890515f701c83aa68 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 18:34:44.950944 28562 raft_consensus.cc:729] T 7c62cb709efc4f9d865b0945a50b8db2 P 885703862b27424890515f701c83aa68 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 885703862b27424890515f701c83aa68, State: Initialized, Role: FOLLOWER
I20260806 18:34:44.951052 28562 consensus_queue.cc:260] T 7c62cb709efc4f9d865b0945a50b8db2 P 885703862b27424890515f701c83aa68 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "885703862b27424890515f701c83aa68" member_type: VOTER last_known_addr { host: "127.19.194.1" port: 34859 } } peers { permanent_uuid: "4575246688ab47bdb5abf0da5c702767" member_type: VOTER last_known_addr { host: "127.19.194.2" port: 44765 } } peers { permanent_uuid: "63955300c20044878851d5e44e44155c" member_type: VOTER last_known_addr { host: "127.19.194.3" port: 46307 } }
I20260806 18:34:44.951303 28562 ts_tablet_manager.cc:1428] T 7c62cb709efc4f9d865b0945a50b8db2 P 885703862b27424890515f701c83aa68: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260806 18:34:44.951309 28584 ts_tablet_manager.cc:1905] T eff105295272444ca25e169aa77c29fc P 4575246688ab47bdb5abf0da5c702767: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 18:34:44.953859 28584 ts_tablet_manager.cc:1918] T eff105295272444ca25e169aa77c29fc P 4575246688ab47bdb5abf0da5c702767: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 18:34:44.953954 28584 log.cc:1196] T eff105295272444ca25e169aa77c29fc P 4575246688ab47bdb5abf0da5c702767: Deleting WAL directory at /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786041246094603-20232-0/minicluster-data/ts-1/wal/wals/eff105295272444ca25e169aa77c29fc
I20260806 18:34:44.954254 28584 ts_tablet_manager.cc:1939] T eff105295272444ca25e169aa77c29fc P 4575246688ab47bdb5abf0da5c702767: Deleting consensus metadata
I20260806 18:34:44.955287 28021 catalog_manager.cc:4841] TS 4575246688ab47bdb5abf0da5c702767 (127.19.194.2:44765): tablet eff105295272444ca25e169aa77c29fc (table test-table [id=117ef29ae96743be9deb026043e7d921]) successfully deleted
I20260806 18:34:44.976531 28566 raft_consensus.cc:482] T 7c62cb709efc4f9d865b0945a50b8db2 P 4575246688ab47bdb5abf0da5c702767 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 18:34:44.976671 28566 raft_consensus.cc:504] T 7c62cb709efc4f9d865b0945a50b8db2 P 4575246688ab47bdb5abf0da5c702767 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "885703862b27424890515f701c83aa68" member_type: VOTER last_known_addr { host: "127.19.194.1" port: 34859 } } peers { permanent_uuid: "4575246688ab47bdb5abf0da5c702767" member_type: VOTER last_known_addr { host: "127.19.194.2" port: 44765 } } peers { permanent_uuid: "63955300c20044878851d5e44e44155c" member_type: VOTER last_known_addr { host: "127.19.194.3" port: 46307 } }
I20260806 18:34:44.977758 28566 leader_election.cc:288] T 7c62cb709efc4f9d865b0945a50b8db2 P 4575246688ab47bdb5abf0da5c702767 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 885703862b27424890515f701c83aa68 (127.19.194.1:34859), 63955300c20044878851d5e44e44155c (127.19.194.3:46307)
W20260806 18:34:44.979815 28389 tablet.cc:2369] T 7c62cb709efc4f9d865b0945a50b8db2 P 4575246688ab47bdb5abf0da5c702767: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 18:34:44.982343 28589 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:34:44.977998 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.194.3:46307 (local address 127.19.194.2:56979)
0806 18:34:44.978324 (+ 326us) negotiation.cc:107] Waiting for socket to connect
0806 18:34:44.978336 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0806 18:34:44.978412 (+ 76us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:34:44.978710 (+ 298us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:34:44.978712 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:34:44.978725 (+ 13us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 18:34:44.978843 (+ 118us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:34:44.978849 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:34:44.979649 (+ 800us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:34:44.979654 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:34:44.980793 (+ 1139us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:34:44.980804 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:34:44.981814 (+ 1010us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:34:44.981818 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:34:44.981984 (+ 166us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:34:44.982013 (+ 29us) client_negotiation.cc:769] Sending connection context
0806 18:34:44.982107 (+ 94us) client_negotiation.cc:240] Negotiation successful
0806 18:34:44.982178 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":268,"mutex_wait_us":5,"thread_start_us":89,"threads_started":1}
I20260806 18:34:44.982352 28555 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:34:44.978353 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.194.2:56979 (local address 127.19.194.3:46307)
0806 18:34:44.978466 (+ 113us) server_negotiation.cc:206] Beginning negotiation
0806 18:34:44.978470 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 18:34:44.978483 (+ 13us) server_negotiation.cc:407] Connection header received
0806 18:34:44.978540 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:34:44.978543 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:34:44.978584 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:34:44.978687 (+ 103us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 18:34:44.978963 (+ 276us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:34:44.979535 (+ 572us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:34:44.980976 (+ 1441us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:34:44.981678 (+ 702us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:34:44.981794 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:34:44.981866 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0806 18:34:44.982144 (+ 278us) server_negotiation.cc:299] Negotiation successful
0806 18:34:44.982221 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260806 18:34:44.982630 28558 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:34:44.978041 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.194.2:37957 (local address 127.19.194.1:34859)
0806 18:34:44.978154 (+ 113us) server_negotiation.cc:206] Beginning negotiation
0806 18:34:44.978158 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 18:34:44.978171 (+ 13us) server_negotiation.cc:407] Connection header received
0806 18:34:44.978299 (+ 128us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:34:44.978301 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:34:44.978356 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:34:44.978423 (+ 67us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 18:34:44.978847 (+ 424us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:34:44.979676 (+ 829us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:34:44.981210 (+ 1534us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:34:44.981808 (+ 598us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:34:44.981919 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:34:44.981977 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0806 18:34:44.982400 (+ 423us) server_negotiation.cc:299] Negotiation successful
0806 18:34:44.982490 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260806 18:34:44.982900 28272 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:34:44.977909 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.194.1:34859 (local address 127.19.194.2:37957)
0806 18:34:44.978074 (+ 165us) negotiation.cc:107] Waiting for socket to connect
0806 18:34:44.978090 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0806 18:34:44.978174 (+ 84us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:34:44.978560 (+ 386us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:34:44.978563 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:34:44.978586 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 18:34:44.978728 (+ 142us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:34:44.978735 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:34:44.979814 (+ 1079us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:34:44.979817 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:34:44.981068 (+ 1251us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:34:44.981080 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:34:44.981943 (+ 863us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:34:44.981947 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:34:44.982171 (+ 224us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:34:44.982193 (+ 22us) client_negotiation.cc:769] Sending connection context
0806 18:34:44.982267 (+ 74us) client_negotiation.cc:240] Negotiation successful
0806 18:34:44.982336 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":90,"mutex_wait_us":83}
I20260806 18:34:44.982888 28465 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7c62cb709efc4f9d865b0945a50b8db2" candidate_uuid: "4575246688ab47bdb5abf0da5c702767" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "63955300c20044878851d5e44e44155c" is_pre_election: true
I20260806 18:34:44.983053 28465 raft_consensus.cc:2454] T 7c62cb709efc4f9d865b0945a50b8db2 P 63955300c20044878851d5e44e44155c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4575246688ab47bdb5abf0da5c702767 in term 0.
I20260806 18:34:44.983453 28264 leader_election.cc:302] T 7c62cb709efc4f9d865b0945a50b8db2 P 4575246688ab47bdb5abf0da5c702767 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4575246688ab47bdb5abf0da5c702767, 63955300c20044878851d5e44e44155c; no voters:
I20260806 18:34:44.983444 28193 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7c62cb709efc4f9d865b0945a50b8db2" candidate_uuid: "4575246688ab47bdb5abf0da5c702767" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "885703862b27424890515f701c83aa68" is_pre_election: true
I20260806 18:34:44.983672 28193 raft_consensus.cc:2454] T 7c62cb709efc4f9d865b0945a50b8db2 P 885703862b27424890515f701c83aa68 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4575246688ab47bdb5abf0da5c702767 in term 0.
I20260806 18:34:44.983722 28566 raft_consensus.cc:2781] T 7c62cb709efc4f9d865b0945a50b8db2 P 4575246688ab47bdb5abf0da5c702767 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 18:34:44.983769 28566 raft_consensus.cc:482] T 7c62cb709efc4f9d865b0945a50b8db2 P 4575246688ab47bdb5abf0da5c702767 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 18:34:44.983826 28566 raft_consensus.cc:3037] T 7c62cb709efc4f9d865b0945a50b8db2 P 4575246688ab47bdb5abf0da5c702767 [term 0 FOLLOWER]: Advancing to term 1
I20260806 18:34:44.985015 28566 raft_consensus.cc:504] T 7c62cb709efc4f9d865b0945a50b8db2 P 4575246688ab47bdb5abf0da5c702767 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "885703862b27424890515f701c83aa68" member_type: VOTER last_known_addr { host: "127.19.194.1" port: 34859 } } peers { permanent_uuid: "4575246688ab47bdb5abf0da5c702767" member_type: VOTER last_known_addr { host: "127.19.194.2" port: 44765 } } peers { permanent_uuid: "63955300c20044878851d5e44e44155c" member_type: VOTER last_known_addr { host: "127.19.194.3" port: 46307 } }
I20260806 18:34:44.985373 28566 leader_election.cc:288] T 7c62cb709efc4f9d865b0945a50b8db2 P 4575246688ab47bdb5abf0da5c702767 [CANDIDATE]: Term 1 election: Requested vote from peers 885703862b27424890515f701c83aa68 (127.19.194.1:34859), 63955300c20044878851d5e44e44155c (127.19.194.3:46307)
I20260806 18:34:44.985725 28465 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7c62cb709efc4f9d865b0945a50b8db2" candidate_uuid: "4575246688ab47bdb5abf0da5c702767" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "63955300c20044878851d5e44e44155c"
I20260806 18:34:44.985853 28465 raft_consensus.cc:3037] T 7c62cb709efc4f9d865b0945a50b8db2 P 63955300c20044878851d5e44e44155c [term 0 FOLLOWER]: Advancing to term 1
I20260806 18:34:44.985888 28193 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7c62cb709efc4f9d865b0945a50b8db2" candidate_uuid: "4575246688ab47bdb5abf0da5c702767" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "885703862b27424890515f701c83aa68"
I20260806 18:34:44.986003 28193 raft_consensus.cc:3037] T 7c62cb709efc4f9d865b0945a50b8db2 P 885703862b27424890515f701c83aa68 [term 0 FOLLOWER]: Advancing to term 1
I20260806 18:34:44.986747 28465 raft_consensus.cc:2454] T 7c62cb709efc4f9d865b0945a50b8db2 P 63955300c20044878851d5e44e44155c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4575246688ab47bdb5abf0da5c702767 in term 1.
I20260806 18:34:44.987028 28264 leader_election.cc:302] T 7c62cb709efc4f9d865b0945a50b8db2 P 4575246688ab47bdb5abf0da5c702767 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4575246688ab47bdb5abf0da5c702767, 63955300c20044878851d5e44e44155c; no voters:
I20260806 18:34:44.987207 28193 raft_consensus.cc:2454] T 7c62cb709efc4f9d865b0945a50b8db2 P 885703862b27424890515f701c83aa68 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4575246688ab47bdb5abf0da5c702767 in term 1.
I20260806 18:34:44.987272 28566 raft_consensus.cc:2781] T 7c62cb709efc4f9d865b0945a50b8db2 P 4575246688ab47bdb5abf0da5c702767 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 18:34:44.987478 28566 raft_consensus.cc:686] T 7c62cb709efc4f9d865b0945a50b8db2 P 4575246688ab47bdb5abf0da5c702767 [term 1 LEADER]: Becoming Leader. State: Replica: 4575246688ab47bdb5abf0da5c702767, State: Running, Role: LEADER
I20260806 18:34:44.987722 28566 consensus_queue.cc:237] T 7c62cb709efc4f9d865b0945a50b8db2 P 4575246688ab47bdb5abf0da5c702767 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "885703862b27424890515f701c83aa68" member_type: VOTER last_known_addr { host: "127.19.194.1" port: 34859 } } peers { permanent_uuid: "4575246688ab47bdb5abf0da5c702767" member_type: VOTER last_known_addr { host: "127.19.194.2" port: 44765 } } peers { permanent_uuid: "63955300c20044878851d5e44e44155c" member_type: VOTER last_known_addr { host: "127.19.194.3" port: 46307 } }
I20260806 18:34:44.989892 28033 catalog_manager.cc:5495] T 7c62cb709efc4f9d865b0945a50b8db2 P 4575246688ab47bdb5abf0da5c702767 reported cstate change: term changed from 0 to 1, leader changed from <none> to 4575246688ab47bdb5abf0da5c702767 (127.19.194.2). New cstate: current_term: 1 leader_uuid: "4575246688ab47bdb5abf0da5c702767" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "885703862b27424890515f701c83aa68" member_type: VOTER last_known_addr { host: "127.19.194.1" port: 34859 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4575246688ab47bdb5abf0da5c702767" member_type: VOTER last_known_addr { host: "127.19.194.2" port: 44765 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "63955300c20044878851d5e44e44155c" member_type: VOTER last_known_addr { host: "127.19.194.3" port: 46307 } health_report { overall_health: UNKNOWN } } }
W20260806 18:34:45.009537 28240 tablet.cc:2369] T 7c62cb709efc4f9d865b0945a50b8db2 P 885703862b27424890515f701c83aa68: Can't schedule compaction. Clean time has not been advanced past its initial value.
Aug 06 18:34:45 dist-test-slave-1xrh krb5kdc[27989](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041283, etypes {rep=17 tkt=17 ses=17}, oryx/127.19.194.62@KRBTEST.COM for oryx/127.19.194.62@KRBTEST.COM
I20260806 18:34:45.034561 28085 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:34:45.025147 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46802 (local address 127.19.194.62:33607)
0806 18:34:45.025287 (+ 140us) server_negotiation.cc:206] Beginning negotiation
0806 18:34:45.025292 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 18:34:45.025461 (+ 169us) server_negotiation.cc:407] Connection header received
0806 18:34:45.025608 (+ 147us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:34:45.025611 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:34:45.025664 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:34:45.025732 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:34:45.026561 (+ 829us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:34:45.027217 (+ 656us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:34:45.028364 (+ 1147us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:34:45.028562 (+ 198us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:34:45.028640 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:34:45.031247 (+ 2607us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:34:45.031251 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:34:45.031253 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 18:34:45.031272 (+ 19us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:34:45.032703 (+ 1431us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:34:45.033228 (+ 525us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:34:45.033231 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:34:45.033233 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:34:45.033356 (+ 123us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:34:45.033641 (+ 285us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:34:45.033644 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:34:45.033646 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:34:45.033824 (+ 178us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:34:45.033931 (+ 107us) server_negotiation.cc:1035] Waiting for connection context
0806 18:34:45.034220 (+ 289us) server_negotiation.cc:299] Negotiation successful
0806 18:34:45.034423 (+ 203us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":96,"server-negotiator.queue_time_us":58}
I20260806 18:34:45.034585 28597 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:34:45.025079 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.194.62:33607 (local address 127.0.0.1:46802)
0806 18:34:45.025363 (+ 284us) negotiation.cc:107] Waiting for socket to connect
0806 18:34:45.025379 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0806 18:34:45.025518 (+ 139us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:34:45.025786 (+ 268us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:34:45.025789 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:34:45.026311 (+ 522us) client_negotiation.cc:189] Negotiated authn=SASL
0806 18:34:45.026434 (+ 123us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:34:45.026442 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:34:45.027367 (+ 925us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:34:45.027370 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:34:45.028226 (+ 856us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:34:45.028236 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:34:45.028683 (+ 447us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:34:45.028687 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:34:45.028807 (+ 120us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:34:45.029398 (+ 591us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 18:34:45.029413 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0806 18:34:45.031091 (+ 1678us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 18:34:45.032877 (+ 1786us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:34:45.032888 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:34:45.032894 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:34:45.033115 (+ 221us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:34:45.033452 (+ 337us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:34:45.033456 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:34:45.033458 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:34:45.033544 (+ 86us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:34:45.033940 (+ 396us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 18:34:45.033949 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 18:34:45.034056 (+ 107us) client_negotiation.cc:769] Sending connection context
0806 18:34:45.034174 (+ 118us) client_negotiation.cc:240] Negotiation successful
0806 18:34:45.034340 (+ 166us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":190,"thread_start_us":101,"threads_started":1}
I20260806 18:34:45.039179 28033 catalog_manager.cc:1908] Servicing CreateTable request from {username='oryx', principal='oryx/127.19.194.62@KRBTEST.COM'} at 127.0.0.1:46802:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260806 18:34:45.040004 28033 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 18:34:45.045137 28173 tablet_service.cc:1467] Processing CreateTablet for tablet a4c01d9a62d340d7a496e09120b089e4 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=674cc4c5cead40f99dea0c5de825c30f]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260806 18:34:45.045364 28462 tablet_service.cc:1467] Processing CreateTablet for tablet a4c01d9a62d340d7a496e09120b089e4 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=674cc4c5cead40f99dea0c5de825c30f]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260806 18:34:45.045573 28173 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a4c01d9a62d340d7a496e09120b089e4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 18:34:45.045763 28462 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a4c01d9a62d340d7a496e09120b089e4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 18:34:45.048454 28560 tablet_bootstrap.cc:492] T a4c01d9a62d340d7a496e09120b089e4 P 63955300c20044878851d5e44e44155c: Bootstrap starting.
I20260806 18:34:45.048920 28313 tablet_service.cc:1467] Processing CreateTablet for tablet a4c01d9a62d340d7a496e09120b089e4 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=674cc4c5cead40f99dea0c5de825c30f]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260806 18:34:45.049343 28313 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a4c01d9a62d340d7a496e09120b089e4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 18:34:45.049502 28560 tablet_bootstrap.cc:654] T a4c01d9a62d340d7a496e09120b089e4 P 63955300c20044878851d5e44e44155c: Neither blocks nor log segments found. Creating new log.
I20260806 18:34:45.050494 28560 tablet_bootstrap.cc:492] T a4c01d9a62d340d7a496e09120b089e4 P 63955300c20044878851d5e44e44155c: No bootstrap required, opened a new log
I20260806 18:34:45.050595 28560 ts_tablet_manager.cc:1397] T a4c01d9a62d340d7a496e09120b089e4 P 63955300c20044878851d5e44e44155c: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.002s
I20260806 18:34:45.051021 28560 raft_consensus.cc:348] T a4c01d9a62d340d7a496e09120b089e4 P 63955300c20044878851d5e44e44155c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "885703862b27424890515f701c83aa68" member_type: VOTER last_known_addr { host: "127.19.194.1" port: 34859 } } peers { permanent_uuid: "4575246688ab47bdb5abf0da5c702767" member_type: VOTER last_known_addr { host: "127.19.194.2" port: 44765 } } peers { permanent_uuid: "63955300c20044878851d5e44e44155c" member_type: VOTER last_known_addr { host: "127.19.194.3" port: 46307 } }
I20260806 18:34:45.051174 28560 raft_consensus.cc:374] T a4c01d9a62d340d7a496e09120b089e4 P 63955300c20044878851d5e44e44155c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 18:34:45.051213 28560 raft_consensus.cc:729] T a4c01d9a62d340d7a496e09120b089e4 P 63955300c20044878851d5e44e44155c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 63955300c20044878851d5e44e44155c, State: Initialized, Role: FOLLOWER
I20260806 18:34:45.051339 28560 consensus_queue.cc:260] T a4c01d9a62d340d7a496e09120b089e4 P 63955300c20044878851d5e44e44155c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "885703862b27424890515f701c83aa68" member_type: VOTER last_known_addr { host: "127.19.194.1" port: 34859 } } peers { permanent_uuid: "4575246688ab47bdb5abf0da5c702767" member_type: VOTER last_known_addr { host: "127.19.194.2" port: 44765 } } peers { permanent_uuid: "63955300c20044878851d5e44e44155c" member_type: VOTER last_known_addr { host: "127.19.194.3" port: 46307 } }
I20260806 18:34:45.051610 28560 ts_tablet_manager.cc:1428] T a4c01d9a62d340d7a496e09120b089e4 P 63955300c20044878851d5e44e44155c: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260806 18:34:45.051843 28565 tablet_replica.cc:439] T a4c01d9a62d340d7a496e09120b089e4 P 63955300c20044878851d5e44e44155c: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "885703862b27424890515f701c83aa68" member_type: VOTER last_known_addr { host: "127.19.194.1" port: 34859 } } peers { permanent_uuid: "4575246688ab47bdb5abf0da5c702767" member_type: VOTER last_known_addr { host: "127.19.194.2" port: 44765 } } peers { permanent_uuid: "63955300c20044878851d5e44e44155c" member_type: VOTER last_known_addr { host: "127.19.194.3" port: 46307 } } }
I20260806 18:34:45.051980 28565 tablet_replica.cc:442] T a4c01d9a62d340d7a496e09120b089e4 P 63955300c20044878851d5e44e44155c: This TxnStatusTablet replica's current role is: FOLLOWER
I20260806 18:34:45.052459 28559 tablet_bootstrap.cc:492] T a4c01d9a62d340d7a496e09120b089e4 P 4575246688ab47bdb5abf0da5c702767: Bootstrap starting.
I20260806 18:34:45.052757 28562 tablet_bootstrap.cc:492] T a4c01d9a62d340d7a496e09120b089e4 P 885703862b27424890515f701c83aa68: Bootstrap starting.
I20260806 18:34:45.053406 28559 tablet_bootstrap.cc:654] T a4c01d9a62d340d7a496e09120b089e4 P 4575246688ab47bdb5abf0da5c702767: Neither blocks nor log segments found. Creating new log.
I20260806 18:34:45.053840 28562 tablet_bootstrap.cc:654] T a4c01d9a62d340d7a496e09120b089e4 P 885703862b27424890515f701c83aa68: Neither blocks nor log segments found. Creating new log.
I20260806 18:34:45.055178 28562 tablet_bootstrap.cc:492] T a4c01d9a62d340d7a496e09120b089e4 P 885703862b27424890515f701c83aa68: No bootstrap required, opened a new log
I20260806 18:34:45.055269 28562 ts_tablet_manager.cc:1397] T a4c01d9a62d340d7a496e09120b089e4 P 885703862b27424890515f701c83aa68: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260806 18:34:45.055526 28559 tablet_bootstrap.cc:492] T a4c01d9a62d340d7a496e09120b089e4 P 4575246688ab47bdb5abf0da5c702767: No bootstrap required, opened a new log
I20260806 18:34:45.055626 28559 ts_tablet_manager.cc:1397] T a4c01d9a62d340d7a496e09120b089e4 P 4575246688ab47bdb5abf0da5c702767: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260806 18:34:45.055684 28562 raft_consensus.cc:348] T a4c01d9a62d340d7a496e09120b089e4 P 885703862b27424890515f701c83aa68 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "885703862b27424890515f701c83aa68" member_type: VOTER last_known_addr { host: "127.19.194.1" port: 34859 } } peers { permanent_uuid: "4575246688ab47bdb5abf0da5c702767" member_type: VOTER last_known_addr { host: "127.19.194.2" port: 44765 } } peers { permanent_uuid: "63955300c20044878851d5e44e44155c" member_type: VOTER last_known_addr { host: "127.19.194.3" port: 46307 } }
I20260806 18:34:45.055810 28562 raft_consensus.cc:374] T a4c01d9a62d340d7a496e09120b089e4 P 885703862b27424890515f701c83aa68 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 18:34:45.055850 28562 raft_consensus.cc:729] T a4c01d9a62d340d7a496e09120b089e4 P 885703862b27424890515f701c83aa68 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 885703862b27424890515f701c83aa68, State: Initialized, Role: FOLLOWER
I20260806 18:34:45.055979 28562 consensus_queue.cc:260] T a4c01d9a62d340d7a496e09120b089e4 P 885703862b27424890515f701c83aa68 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "885703862b27424890515f701c83aa68" member_type: VOTER last_known_addr { host: "127.19.194.1" port: 34859 } } peers { permanent_uuid: "4575246688ab47bdb5abf0da5c702767" member_type: VOTER last_known_addr { host: "127.19.194.2" port: 44765 } } peers { permanent_uuid: "63955300c20044878851d5e44e44155c" member_type: VOTER last_known_addr { host: "127.19.194.3" port: 46307 } }
I20260806 18:34:45.056064 28559 raft_consensus.cc:348] T a4c01d9a62d340d7a496e09120b089e4 P 4575246688ab47bdb5abf0da5c702767 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "885703862b27424890515f701c83aa68" member_type: VOTER last_known_addr { host: "127.19.194.1" port: 34859 } } peers { permanent_uuid: "4575246688ab47bdb5abf0da5c702767" member_type: VOTER last_known_addr { host: "127.19.194.2" port: 44765 } } peers { permanent_uuid: "63955300c20044878851d5e44e44155c" member_type: VOTER last_known_addr { host: "127.19.194.3" port: 46307 } }
I20260806 18:34:45.056200 28559 raft_consensus.cc:374] T a4c01d9a62d340d7a496e09120b089e4 P 4575246688ab47bdb5abf0da5c702767 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 18:34:45.056238 28559 raft_consensus.cc:729] T a4c01d9a62d340d7a496e09120b089e4 P 4575246688ab47bdb5abf0da5c702767 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4575246688ab47bdb5abf0da5c702767, State: Initialized, Role: FOLLOWER
I20260806 18:34:45.056243 28578 tablet_replica.cc:439] T a4c01d9a62d340d7a496e09120b089e4 P 885703862b27424890515f701c83aa68: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "885703862b27424890515f701c83aa68" member_type: VOTER last_known_addr { host: "127.19.194.1" port: 34859 } } peers { permanent_uuid: "4575246688ab47bdb5abf0da5c702767" member_type: VOTER last_known_addr { host: "127.19.194.2" port: 44765 } } peers { permanent_uuid: "63955300c20044878851d5e44e44155c" member_type: VOTER last_known_addr { host: "127.19.194.3" port: 46307 } } }
I20260806 18:34:45.056324 28562 ts_tablet_manager.cc:1428] T a4c01d9a62d340d7a496e09120b089e4 P 885703862b27424890515f701c83aa68: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20260806 18:34:45.056373 28578 tablet_replica.cc:442] T a4c01d9a62d340d7a496e09120b089e4 P 885703862b27424890515f701c83aa68: This TxnStatusTablet replica's current role is: FOLLOWER
I20260806 18:34:45.056408 28559 consensus_queue.cc:260] T a4c01d9a62d340d7a496e09120b089e4 P 4575246688ab47bdb5abf0da5c702767 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "885703862b27424890515f701c83aa68" member_type: VOTER last_known_addr { host: "127.19.194.1" port: 34859 } } peers { permanent_uuid: "4575246688ab47bdb5abf0da5c702767" member_type: VOTER last_known_addr { host: "127.19.194.2" port: 44765 } } peers { permanent_uuid: "63955300c20044878851d5e44e44155c" member_type: VOTER last_known_addr { host: "127.19.194.3" port: 46307 } }
I20260806 18:34:45.056689 28590 tablet_replica.cc:439] T a4c01d9a62d340d7a496e09120b089e4 P 4575246688ab47bdb5abf0da5c702767: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "885703862b27424890515f701c83aa68" member_type: VOTER last_known_addr { host: "127.19.194.1" port: 34859 } } peers { permanent_uuid: "4575246688ab47bdb5abf0da5c702767" member_type: VOTER last_known_addr { host: "127.19.194.2" port: 44765 } } peers { permanent_uuid: "63955300c20044878851d5e44e44155c" member_type: VOTER last_known_addr { host: "127.19.194.3" port: 46307 } } }
I20260806 18:34:45.056846 28590 tablet_replica.cc:442] T a4c01d9a62d340d7a496e09120b089e4 P 4575246688ab47bdb5abf0da5c702767: This TxnStatusTablet replica's current role is: FOLLOWER
I20260806 18:34:45.056706 28559 ts_tablet_manager.cc:1428] T a4c01d9a62d340d7a496e09120b089e4 P 4575246688ab47bdb5abf0da5c702767: Time spent starting tablet: real 0.001s user 0.002s sys 0.000s
I20260806 18:34:45.112280 28578 raft_consensus.cc:482] T a4c01d9a62d340d7a496e09120b089e4 P 885703862b27424890515f701c83aa68 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 18:34:45.112427 28578 raft_consensus.cc:504] T a4c01d9a62d340d7a496e09120b089e4 P 885703862b27424890515f701c83aa68 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "885703862b27424890515f701c83aa68" member_type: VOTER last_known_addr { host: "127.19.194.1" port: 34859 } } peers { permanent_uuid: "4575246688ab47bdb5abf0da5c702767" member_type: VOTER last_known_addr { host: "127.19.194.2" port: 44765 } } peers { permanent_uuid: "63955300c20044878851d5e44e44155c" member_type: VOTER last_known_addr { host: "127.19.194.3" port: 46307 } }
I20260806 18:34:45.112829 28578 leader_election.cc:288] T a4c01d9a62d340d7a496e09120b089e4 P 885703862b27424890515f701c83aa68 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4575246688ab47bdb5abf0da5c702767 (127.19.194.2:44765), 63955300c20044878851d5e44e44155c (127.19.194.3:46307)
I20260806 18:34:45.113341 28334 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a4c01d9a62d340d7a496e09120b089e4" candidate_uuid: "885703862b27424890515f701c83aa68" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4575246688ab47bdb5abf0da5c702767" is_pre_election: true
I20260806 18:34:45.113338 28465 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a4c01d9a62d340d7a496e09120b089e4" candidate_uuid: "885703862b27424890515f701c83aa68" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "63955300c20044878851d5e44e44155c" is_pre_election: true
I20260806 18:34:45.113505 28465 raft_consensus.cc:2454] T a4c01d9a62d340d7a496e09120b089e4 P 63955300c20044878851d5e44e44155c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 885703862b27424890515f701c83aa68 in term 0.
I20260806 18:34:45.113505 28334 raft_consensus.cc:2454] T a4c01d9a62d340d7a496e09120b089e4 P 4575246688ab47bdb5abf0da5c702767 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 885703862b27424890515f701c83aa68 in term 0.
I20260806 18:34:45.113858 28124 leader_election.cc:302] T a4c01d9a62d340d7a496e09120b089e4 P 885703862b27424890515f701c83aa68 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 4575246688ab47bdb5abf0da5c702767, 63955300c20044878851d5e44e44155c, 885703862b27424890515f701c83aa68; no voters:
I20260806 18:34:45.114048 28578 raft_consensus.cc:2781] T a4c01d9a62d340d7a496e09120b089e4 P 885703862b27424890515f701c83aa68 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 18:34:45.114125 28578 raft_consensus.cc:482] T a4c01d9a62d340d7a496e09120b089e4 P 885703862b27424890515f701c83aa68 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 18:34:45.114167 28578 raft_consensus.cc:3037] T a4c01d9a62d340d7a496e09120b089e4 P 885703862b27424890515f701c83aa68 [term 0 FOLLOWER]: Advancing to term 1
I20260806 18:34:45.115252 28578 raft_consensus.cc:504] T a4c01d9a62d340d7a496e09120b089e4 P 885703862b27424890515f701c83aa68 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "885703862b27424890515f701c83aa68" member_type: VOTER last_known_addr { host: "127.19.194.1" port: 34859 } } peers { permanent_uuid: "4575246688ab47bdb5abf0da5c702767" member_type: VOTER last_known_addr { host: "127.19.194.2" port: 44765 } } peers { permanent_uuid: "63955300c20044878851d5e44e44155c" member_type: VOTER last_known_addr { host: "127.19.194.3" port: 46307 } }
I20260806 18:34:45.115640 28578 leader_election.cc:288] T a4c01d9a62d340d7a496e09120b089e4 P 885703862b27424890515f701c83aa68 [CANDIDATE]: Term 1 election: Requested vote from peers 4575246688ab47bdb5abf0da5c702767 (127.19.194.2:44765), 63955300c20044878851d5e44e44155c (127.19.194.3:46307)
I20260806 18:34:45.115976 28465 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a4c01d9a62d340d7a496e09120b089e4" candidate_uuid: "885703862b27424890515f701c83aa68" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "63955300c20044878851d5e44e44155c"
I20260806 18:34:45.115976 28334 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a4c01d9a62d340d7a496e09120b089e4" candidate_uuid: "885703862b27424890515f701c83aa68" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4575246688ab47bdb5abf0da5c702767"
I20260806 18:34:45.116096 28465 raft_consensus.cc:3037] T a4c01d9a62d340d7a496e09120b089e4 P 63955300c20044878851d5e44e44155c [term 0 FOLLOWER]: Advancing to term 1
I20260806 18:34:45.116096 28334 raft_consensus.cc:3037] T a4c01d9a62d340d7a496e09120b089e4 P 4575246688ab47bdb5abf0da5c702767 [term 0 FOLLOWER]: Advancing to term 1
I20260806 18:34:45.117223 28465 raft_consensus.cc:2454] T a4c01d9a62d340d7a496e09120b089e4 P 63955300c20044878851d5e44e44155c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 885703862b27424890515f701c83aa68 in term 1.
I20260806 18:34:45.117223 28334 raft_consensus.cc:2454] T a4c01d9a62d340d7a496e09120b089e4 P 4575246688ab47bdb5abf0da5c702767 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 885703862b27424890515f701c83aa68 in term 1.
I20260806 18:34:45.117506 28124 leader_election.cc:302] T a4c01d9a62d340d7a496e09120b089e4 P 885703862b27424890515f701c83aa68 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 63955300c20044878851d5e44e44155c, 885703862b27424890515f701c83aa68; no voters:
I20260806 18:34:45.117686 28578 raft_consensus.cc:2781] T a4c01d9a62d340d7a496e09120b089e4 P 885703862b27424890515f701c83aa68 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 18:34:45.117776 28578 raft_consensus.cc:686] T a4c01d9a62d340d7a496e09120b089e4 P 885703862b27424890515f701c83aa68 [term 1 LEADER]: Becoming Leader. State: Replica: 885703862b27424890515f701c83aa68, State: Running, Role: LEADER
I20260806 18:34:45.117895 28578 consensus_queue.cc:237] T a4c01d9a62d340d7a496e09120b089e4 P 885703862b27424890515f701c83aa68 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "885703862b27424890515f701c83aa68" member_type: VOTER last_known_addr { host: "127.19.194.1" port: 34859 } } peers { permanent_uuid: "4575246688ab47bdb5abf0da5c702767" member_type: VOTER last_known_addr { host: "127.19.194.2" port: 44765 } } peers { permanent_uuid: "63955300c20044878851d5e44e44155c" member_type: VOTER last_known_addr { host: "127.19.194.3" port: 46307 } }
I20260806 18:34:45.118583 28570 tablet_replica.cc:439] T a4c01d9a62d340d7a496e09120b089e4 P 885703862b27424890515f701c83aa68: TxnStatusTablet state changed. Reason: New leader 885703862b27424890515f701c83aa68. Latest consensus state: current_term: 1 leader_uuid: "885703862b27424890515f701c83aa68" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "885703862b27424890515f701c83aa68" member_type: VOTER last_known_addr { host: "127.19.194.1" port: 34859 } } peers { permanent_uuid: "4575246688ab47bdb5abf0da5c702767" member_type: VOTER last_known_addr { host: "127.19.194.2" port: 44765 } } peers { permanent_uuid: "63955300c20044878851d5e44e44155c" member_type: VOTER last_known_addr { host: "127.19.194.3" port: 46307 } } }
I20260806 18:34:45.118698 28570 tablet_replica.cc:442] T a4c01d9a62d340d7a496e09120b089e4 P 885703862b27424890515f701c83aa68: This TxnStatusTablet replica's current role is: LEADER
I20260806 18:34:45.119009 28603 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260806 18:34:45.119206 28603 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260806 18:34:45.119669 28033 catalog_manager.cc:5495] T a4c01d9a62d340d7a496e09120b089e4 P 885703862b27424890515f701c83aa68 reported cstate change: term changed from 0 to 1, leader changed from <none> to 885703862b27424890515f701c83aa68 (127.19.194.1). New cstate: current_term: 1 leader_uuid: "885703862b27424890515f701c83aa68" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "885703862b27424890515f701c83aa68" member_type: VOTER last_known_addr { host: "127.19.194.1" port: 34859 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4575246688ab47bdb5abf0da5c702767" member_type: VOTER last_known_addr { host: "127.19.194.2" port: 44765 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "63955300c20044878851d5e44e44155c" member_type: VOTER last_known_addr { host: "127.19.194.3" port: 46307 } health_report { overall_health: UNKNOWN } } }
I20260806 18:34:45.134759 28597 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:34:45.131373 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.194.1:34859 (local address 127.0.0.1:33854)
0806 18:34:45.131631 (+ 258us) negotiation.cc:107] Waiting for socket to connect
0806 18:34:45.131645 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0806 18:34:45.131777 (+ 132us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:34:45.132071 (+ 294us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:34:45.132075 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:34:45.132082 (+ 7us) client_negotiation.cc:189] Negotiated authn=TOKEN
0806 18:34:45.132187 (+ 105us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:34:45.132193 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:34:45.132925 (+ 732us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:34:45.132930 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:34:45.133541 (+ 611us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:34:45.133550 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:34:45.134020 (+ 470us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:34:45.134023 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:34:45.134112 (+ 89us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:34:45.134165 (+ 53us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0806 18:34:45.134482 (+ 317us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0806 18:34:45.134500 (+ 18us) client_negotiation.cc:769] Sending connection context
0806 18:34:45.134563 (+ 63us) client_negotiation.cc:240] Negotiation successful
0806 18:34:45.134643 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":160}
I20260806 18:34:45.134752 28558 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:34:45.131448 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33854 (local address 127.19.194.1:34859)
0806 18:34:45.131593 (+ 145us) server_negotiation.cc:206] Beginning negotiation
0806 18:34:45.131598 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 18:34:45.131716 (+ 118us) server_negotiation.cc:407] Connection header received
0806 18:34:45.131907 (+ 191us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:34:45.131910 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:34:45.131966 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:34:45.132049 (+ 83us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 18:34:45.132305 (+ 256us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:34:45.132810 (+ 505us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:34:45.133668 (+ 858us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:34:45.133889 (+ 221us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:34:45.134018 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:34:45.134316 (+ 298us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 18:34:45.134388 (+ 72us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 18:34:45.134469 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0806 18:34:45.134573 (+ 104us) server_negotiation.cc:299] Negotiation successful
0806 18:34:45.134625 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260806 18:34:45.137244 28334 raft_consensus.cc:1261] T a4c01d9a62d340d7a496e09120b089e4 P 4575246688ab47bdb5abf0da5c702767 [term 1 FOLLOWER]: Refusing update from remote peer 885703862b27424890515f701c83aa68: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 18:34:45.137297 28465 raft_consensus.cc:1261] T a4c01d9a62d340d7a496e09120b089e4 P 63955300c20044878851d5e44e44155c [term 1 FOLLOWER]: Refusing update from remote peer 885703862b27424890515f701c83aa68: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 18:34:45.137691 28570 consensus_queue.cc:1035] T a4c01d9a62d340d7a496e09120b089e4 P 885703862b27424890515f701c83aa68 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4575246688ab47bdb5abf0da5c702767" member_type: VOTER last_known_addr { host: "127.19.194.2" port: 44765 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 18:34:45.137895 28578 consensus_queue.cc:1035] T a4c01d9a62d340d7a496e09120b089e4 P 885703862b27424890515f701c83aa68 [LEADER]: Connected to new peer: Peer: permanent_uuid: "63955300c20044878851d5e44e44155c" member_type: VOTER last_known_addr { host: "127.19.194.3" port: 46307 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 18:34:45.139326 28590 tablet_replica.cc:439] T a4c01d9a62d340d7a496e09120b089e4 P 4575246688ab47bdb5abf0da5c702767: TxnStatusTablet state changed. Reason: New leader 885703862b27424890515f701c83aa68. Latest consensus state: current_term: 1 leader_uuid: "885703862b27424890515f701c83aa68" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "885703862b27424890515f701c83aa68" member_type: VOTER last_known_addr { host: "127.19.194.1" port: 34859 } } peers { permanent_uuid: "4575246688ab47bdb5abf0da5c702767" member_type: VOTER last_known_addr { host: "127.19.194.2" port: 44765 } } peers { permanent_uuid: "63955300c20044878851d5e44e44155c" member_type: VOTER last_known_addr { host: "127.19.194.3" port: 46307 } } }
I20260806 18:34:45.139436 28590 tablet_replica.cc:442] T a4c01d9a62d340d7a496e09120b089e4 P 4575246688ab47bdb5abf0da5c702767: This TxnStatusTablet replica's current role is: FOLLOWER
I20260806 18:34:45.140311 28565 tablet_replica.cc:439] T a4c01d9a62d340d7a496e09120b089e4 P 63955300c20044878851d5e44e44155c: TxnStatusTablet state changed. Reason: New leader 885703862b27424890515f701c83aa68. Latest consensus state: current_term: 1 leader_uuid: "885703862b27424890515f701c83aa68" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "885703862b27424890515f701c83aa68" member_type: VOTER last_known_addr { host: "127.19.194.1" port: 34859 } } peers { permanent_uuid: "4575246688ab47bdb5abf0da5c702767" member_type: VOTER last_known_addr { host: "127.19.194.2" port: 44765 } } peers { permanent_uuid: "63955300c20044878851d5e44e44155c" member_type: VOTER last_known_addr { host: "127.19.194.3" port: 46307 } } }
I20260806 18:34:45.140424 28565 tablet_replica.cc:442] T a4c01d9a62d340d7a496e09120b089e4 P 63955300c20044878851d5e44e44155c: This TxnStatusTablet replica's current role is: FOLLOWER
I20260806 18:34:45.141096 28578 tablet_replica.cc:439] T a4c01d9a62d340d7a496e09120b089e4 P 885703862b27424890515f701c83aa68: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "885703862b27424890515f701c83aa68" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "885703862b27424890515f701c83aa68" member_type: VOTER last_known_addr { host: "127.19.194.1" port: 34859 } } peers { permanent_uuid: "4575246688ab47bdb5abf0da5c702767" member_type: VOTER last_known_addr { host: "127.19.194.2" port: 44765 } } peers { permanent_uuid: "63955300c20044878851d5e44e44155c" member_type: VOTER last_known_addr { host: "127.19.194.3" port: 46307 } } }
I20260806 18:34:45.141215 28578 tablet_replica.cc:442] T a4c01d9a62d340d7a496e09120b089e4 P 885703862b27424890515f701c83aa68: This TxnStatusTablet replica's current role is: LEADER
I20260806 18:34:45.141774 28573 mvcc.cc:205] Tried to move back new op lower bound from 7315625103918190592 to 7315625103844098048. Current Snapshot: MvccSnapshot[applied={T|T < 7315625103918190592 or (T in {7315625103918190592})}]
I20260806 18:34:45.142447 28590 tablet_replica.cc:439] T a4c01d9a62d340d7a496e09120b089e4 P 4575246688ab47bdb5abf0da5c702767: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "885703862b27424890515f701c83aa68" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "885703862b27424890515f701c83aa68" member_type: VOTER last_known_addr { host: "127.19.194.1" port: 34859 } } peers { permanent_uuid: "4575246688ab47bdb5abf0da5c702767" member_type: VOTER last_known_addr { host: "127.19.194.2" port: 44765 } } peers { permanent_uuid: "63955300c20044878851d5e44e44155c" member_type: VOTER last_known_addr { host: "127.19.194.3" port: 46307 } } }
I20260806 18:34:45.142544 28590 tablet_replica.cc:442] T a4c01d9a62d340d7a496e09120b089e4 P 4575246688ab47bdb5abf0da5c702767: This TxnStatusTablet replica's current role is: FOLLOWER
I20260806 18:34:45.142632 28575 mvcc.cc:205] Tried to move back new op lower bound from 7315625103918190592 to 7315625103844098048. Current Snapshot: MvccSnapshot[applied={T|T < 7315625103918190592 or (T in {7315625103918190592})}]
I20260806 18:34:45.144804 28567 tablet_replica.cc:439] T a4c01d9a62d340d7a496e09120b089e4 P 885703862b27424890515f701c83aa68: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "885703862b27424890515f701c83aa68" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "885703862b27424890515f701c83aa68" member_type: VOTER last_known_addr { host: "127.19.194.1" port: 34859 } } peers { permanent_uuid: "4575246688ab47bdb5abf0da5c702767" member_type: VOTER last_known_addr { host: "127.19.194.2" port: 44765 } } peers { permanent_uuid: "63955300c20044878851d5e44e44155c" member_type: VOTER last_known_addr { host: "127.19.194.3" port: 46307 } } }
I20260806 18:34:45.144912 28567 tablet_replica.cc:442] T a4c01d9a62d340d7a496e09120b089e4 P 885703862b27424890515f701c83aa68: This TxnStatusTablet replica's current role is: LEADER
I20260806 18:34:45.145315 28565 tablet_replica.cc:439] T a4c01d9a62d340d7a496e09120b089e4 P 63955300c20044878851d5e44e44155c: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "885703862b27424890515f701c83aa68" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "885703862b27424890515f701c83aa68" member_type: VOTER last_known_addr { host: "127.19.194.1" port: 34859 } } peers { permanent_uuid: "4575246688ab47bdb5abf0da5c702767" member_type: VOTER last_known_addr { host: "127.19.194.2" port: 44765 } } peers { permanent_uuid: "63955300c20044878851d5e44e44155c" member_type: VOTER last_known_addr { host: "127.19.194.3" port: 46307 } } }
I20260806 18:34:45.145431 28565 tablet_replica.cc:442] T a4c01d9a62d340d7a496e09120b089e4 P 63955300c20044878851d5e44e44155c: This TxnStatusTablet replica's current role is: FOLLOWER
I20260806 18:34:45.151144 28557 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:34:45.147636 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51856 (local address 127.19.194.2:44765)
0806 18:34:45.147765 (+ 129us) server_negotiation.cc:206] Beginning negotiation
0806 18:34:45.147769 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 18:34:45.147784 (+ 15us) server_negotiation.cc:407] Connection header received
0806 18:34:45.147883 (+ 99us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:34:45.147886 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:34:45.147960 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:34:45.148061 (+ 101us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 18:34:45.148274 (+ 213us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:34:45.148996 (+ 722us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:34:45.149744 (+ 748us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:34:45.150046 (+ 302us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:34:45.150136 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:34:45.150353 (+ 217us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 18:34:45.150818 (+ 465us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 18:34:45.150925 (+ 107us) server_negotiation.cc:1035] Waiting for connection context
0806 18:34:45.150962 (+ 37us) server_negotiation.cc:299] Negotiation successful
0806 18:34:45.151010 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260806 18:34:45.161396 28558 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:34:45.157806 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.194.2:42597 (local address 127.19.194.1:34859)
0806 18:34:45.157943 (+ 137us) server_negotiation.cc:206] Beginning negotiation
0806 18:34:45.157949 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 18:34:45.158159 (+ 210us) server_negotiation.cc:407] Connection header received
0806 18:34:45.158279 (+ 120us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:34:45.158282 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:34:45.158330 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:34:45.158407 (+ 77us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 18:34:45.158651 (+ 244us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:34:45.159115 (+ 464us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:34:45.160348 (+ 1233us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:34:45.160572 (+ 224us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:34:45.160681 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:34:45.160994 (+ 313us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 18:34:45.161056 (+ 62us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 18:34:45.161130 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0806 18:34:45.161213 (+ 83us) server_negotiation.cc:299] Negotiation successful
0806 18:34:45.161259 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260806 18:34:45.161422 28609 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:34:45.157732 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.194.1:34859 (local address 127.19.194.2:42597)
0806 18:34:45.158080 (+ 348us) negotiation.cc:107] Waiting for socket to connect
0806 18:34:45.158091 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0806 18:34:45.158201 (+ 110us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:34:45.158417 (+ 216us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:34:45.158420 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:34:45.158427 (+ 7us) client_negotiation.cc:189] Negotiated authn=TOKEN
0806 18:34:45.158549 (+ 122us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:34:45.158554 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:34:45.159265 (+ 711us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:34:45.159268 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:34:45.160221 (+ 953us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:34:45.160230 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:34:45.160705 (+ 475us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:34:45.160710 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:34:45.160800 (+ 90us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:34:45.160839 (+ 39us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0806 18:34:45.161138 (+ 299us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0806 18:34:45.161146 (+ 8us) client_negotiation.cc:769] Sending connection context
0806 18:34:45.161204 (+ 58us) client_negotiation.cc:240] Negotiation successful
0806 18:34:45.161275 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":265,"thread_start_us":115,"threads_started":1}
I20260806 18:34:45.168174 28465 raft_consensus.cc:1261] T 7c62cb709efc4f9d865b0945a50b8db2 P 63955300c20044878851d5e44e44155c [term 1 FOLLOWER]: Refusing update from remote peer 4575246688ab47bdb5abf0da5c702767: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 18:34:45.168416 28193 raft_consensus.cc:1261] T 7c62cb709efc4f9d865b0945a50b8db2 P 885703862b27424890515f701c83aa68 [term 1 FOLLOWER]: Refusing update from remote peer 4575246688ab47bdb5abf0da5c702767: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 18:34:45.168591 28566 consensus_queue.cc:1035] T 7c62cb709efc4f9d865b0945a50b8db2 P 4575246688ab47bdb5abf0da5c702767 [LEADER]: Connected to new peer: Peer: permanent_uuid: "63955300c20044878851d5e44e44155c" member_type: VOTER last_known_addr { host: "127.19.194.3" port: 46307 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 18:34:45.168890 28590 consensus_queue.cc:1035] T 7c62cb709efc4f9d865b0945a50b8db2 P 4575246688ab47bdb5abf0da5c702767 [LEADER]: Connected to new peer: Peer: permanent_uuid: "885703862b27424890515f701c83aa68" member_type: VOTER last_known_addr { host: "127.19.194.1" port: 34859 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 18:34:45.175268 28574 mvcc.cc:205] Tried to move back new op lower bound from 7315625104044224512 to 7315625103311826944. Current Snapshot: MvccSnapshot[applied={T|T < 7315625104044224512}]
I20260806 18:34:45.191571 28613 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:34:45.187506 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.194.2:44765 (local address 127.19.194.1:46777)
0806 18:34:45.187868 (+ 362us) negotiation.cc:107] Waiting for socket to connect
0806 18:34:45.187884 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0806 18:34:45.188019 (+ 135us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:34:45.188350 (+ 331us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:34:45.188353 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:34:45.188360 (+ 7us) client_negotiation.cc:189] Negotiated authn=TOKEN
0806 18:34:45.188453 (+ 93us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:34:45.188458 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:34:45.189429 (+ 971us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:34:45.189432 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:34:45.190369 (+ 937us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:34:45.190380 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:34:45.190818 (+ 438us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:34:45.190822 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:34:45.190907 (+ 85us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:34:45.190957 (+ 50us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0806 18:34:45.191291 (+ 334us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0806 18:34:45.191300 (+ 9us) client_negotiation.cc:769] Sending connection context
0806 18:34:45.191367 (+ 67us) client_negotiation.cc:240] Negotiation successful
0806 18:34:45.191447 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":238,"thread_start_us":130,"threads_started":1}
I20260806 18:34:45.191567 28557 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:34:45.187573 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.194.1:46777 (local address 127.19.194.2:44765)
0806 18:34:45.187713 (+ 140us) server_negotiation.cc:206] Beginning negotiation
0806 18:34:45.187718 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 18:34:45.187948 (+ 230us) server_negotiation.cc:407] Connection header received
0806 18:34:45.188127 (+ 179us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:34:45.188130 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:34:45.188173 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:34:45.188216 (+ 43us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 18:34:45.188563 (+ 347us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:34:45.189285 (+ 722us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:34:45.190491 (+ 1206us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:34:45.190707 (+ 216us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:34:45.190790 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:34:45.191076 (+ 286us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 18:34:45.191187 (+ 111us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 18:34:45.191284 (+ 97us) server_negotiation.cc:1035] Waiting for connection context
0806 18:34:45.191379 (+ 95us) server_negotiation.cc:299] Negotiation successful
0806 18:34:45.191439 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260806 18:34:45.218178 28035 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:46786:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 18:34:45.218374 28035 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:46786:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 18:34:45.220708 28035 catalog_manager.cc:5782] T 00000000000000000000000000000000 P d0282f2efb864823bbf90a21992ab39d: Sending DeleteTablet for 3 replicas of tablet 7c62cb709efc4f9d865b0945a50b8db2
I20260806 18:34:45.221218 28173 tablet_service.cc:1514] Processing DeleteTablet for tablet 7c62cb709efc4f9d865b0945a50b8db2 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 18:34:45 UTC) from {username='oryx', principal='oryx/127.19.194.62@KRBTEST.COM'} at 127.0.0.1:33838
I20260806 18:34:45.221426 28585 tablet_replica.cc:330] T 7c62cb709efc4f9d865b0945a50b8db2 P 885703862b27424890515f701c83aa68: stopping tablet replica
I20260806 18:34:45.221467 28462 tablet_service.cc:1514] Processing DeleteTablet for tablet 7c62cb709efc4f9d865b0945a50b8db2 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 18:34:45 UTC) from {username='oryx', principal='oryx/127.19.194.62@KRBTEST.COM'} at 127.0.0.1:41986
I20260806 18:34:45.221583 28585 raft_consensus.cc:2229] T 7c62cb709efc4f9d865b0945a50b8db2 P 885703862b27424890515f701c83aa68 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 18:34:45.221660 28583 tablet_replica.cc:330] T 7c62cb709efc4f9d865b0945a50b8db2 P 63955300c20044878851d5e44e44155c: stopping tablet replica
I20260806 18:34:45.221730 28585 raft_consensus.cc:2258] T 7c62cb709efc4f9d865b0945a50b8db2 P 885703862b27424890515f701c83aa68 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 18:34:45.221776 28583 raft_consensus.cc:2229] T 7c62cb709efc4f9d865b0945a50b8db2 P 63955300c20044878851d5e44e44155c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 18:34:45.221886 28583 raft_consensus.cc:2258] T 7c62cb709efc4f9d865b0945a50b8db2 P 63955300c20044878851d5e44e44155c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 18:34:45.222659 28585 ts_tablet_manager.cc:1905] T 7c62cb709efc4f9d865b0945a50b8db2 P 885703862b27424890515f701c83aa68: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 18:34:45.223031 28583 ts_tablet_manager.cc:1905] T 7c62cb709efc4f9d865b0945a50b8db2 P 63955300c20044878851d5e44e44155c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 18:34:45.225667 28585 ts_tablet_manager.cc:1918] T 7c62cb709efc4f9d865b0945a50b8db2 P 885703862b27424890515f701c83aa68: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260806 18:34:45.225736 28585 log.cc:1196] T 7c62cb709efc4f9d865b0945a50b8db2 P 885703862b27424890515f701c83aa68: Deleting WAL directory at /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786041246094603-20232-0/minicluster-data/ts-0/wal/wals/7c62cb709efc4f9d865b0945a50b8db2
I20260806 18:34:45.226102 28583 ts_tablet_manager.cc:1918] T 7c62cb709efc4f9d865b0945a50b8db2 P 63955300c20044878851d5e44e44155c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260806 18:34:45.226130 28585 ts_tablet_manager.cc:1939] T 7c62cb709efc4f9d865b0945a50b8db2 P 885703862b27424890515f701c83aa68: Deleting consensus metadata
I20260806 18:34:45.226179 28583 log.cc:1196] T 7c62cb709efc4f9d865b0945a50b8db2 P 63955300c20044878851d5e44e44155c: Deleting WAL directory at /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786041246094603-20232-0/minicluster-data/ts-2/wal/wals/7c62cb709efc4f9d865b0945a50b8db2
I20260806 18:34:45.226707 28313 tablet_service.cc:1514] Processing DeleteTablet for tablet 7c62cb709efc4f9d865b0945a50b8db2 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 18:34:45 UTC) from {username='oryx', principal='oryx/127.19.194.62@KRBTEST.COM'} at 127.0.0.1:51842
I20260806 18:34:45.226938 28584 tablet_replica.cc:330] T 7c62cb709efc4f9d865b0945a50b8db2 P 4575246688ab47bdb5abf0da5c702767: stopping tablet replica
I20260806 18:34:45.227053 28584 raft_consensus.cc:2229] T 7c62cb709efc4f9d865b0945a50b8db2 P 4575246688ab47bdb5abf0da5c702767 [term 1 LEADER]: Raft consensus shutting down.
I20260806 18:34:45.227406 28584 raft_consensus.cc:2258] T 7c62cb709efc4f9d865b0945a50b8db2 P 4575246688ab47bdb5abf0da5c702767 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 18:34:45.229773 28584 ts_tablet_manager.cc:1905] T 7c62cb709efc4f9d865b0945a50b8db2 P 4575246688ab47bdb5abf0da5c702767: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 18:34:45.230696 28022 catalog_manager.cc:4841] TS 885703862b27424890515f701c83aa68 (127.19.194.1:34859): tablet 7c62cb709efc4f9d865b0945a50b8db2 (table test-table [id=b0a995d8f6204467b57e7d329e46fc68]) successfully deleted
I20260806 18:34:45.232530 28584 ts_tablet_manager.cc:1918] T 7c62cb709efc4f9d865b0945a50b8db2 P 4575246688ab47bdb5abf0da5c702767: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260806 18:34:45.232604 28584 log.cc:1196] T 7c62cb709efc4f9d865b0945a50b8db2 P 4575246688ab47bdb5abf0da5c702767: Deleting WAL directory at /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786041246094603-20232-0/minicluster-data/ts-1/wal/wals/7c62cb709efc4f9d865b0945a50b8db2
I20260806 18:34:45.232856 28584 ts_tablet_manager.cc:1939] T 7c62cb709efc4f9d865b0945a50b8db2 P 4575246688ab47bdb5abf0da5c702767: Deleting consensus metadata
I20260806 18:34:45.233530 28021 catalog_manager.cc:4841] TS 4575246688ab47bdb5abf0da5c702767 (127.19.194.2:44765): tablet 7c62cb709efc4f9d865b0945a50b8db2 (table test-table [id=b0a995d8f6204467b57e7d329e46fc68]) successfully deleted
I20260806 18:34:45.234642 28583 ts_tablet_manager.cc:1939] T 7c62cb709efc4f9d865b0945a50b8db2 P 63955300c20044878851d5e44e44155c: Deleting consensus metadata
I20260806 18:34:45.235383 28019 catalog_manager.cc:4841] TS 63955300c20044878851d5e44e44155c (127.19.194.3:46307): tablet 7c62cb709efc4f9d865b0945a50b8db2 (table test-table [id=b0a995d8f6204467b57e7d329e46fc68]) successfully deleted
Aug 06 18:34:45 dist-test-slave-1xrh krb5kdc[27989](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041285, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 06 18:34:45 dist-test-slave-1xrh krb5kdc[27989](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-admin@KRBTEST.COM for kudu/127.19.194.62@KRBTEST.COM, Server not found in Kerberos database
Aug 06 18:34:45 dist-test-slave-1xrh krb5kdc[27989](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-admin@KRBTEST.COM for kudu/127.19.194.62@KRBTEST.COM, Server not found in Kerberos database
W20260806 18:34:45.258534 28085 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 18:34:45.251977 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46812 (local address 127.19.194.62:33607)
0806 18:34:45.252105 (+ 128us) server_negotiation.cc:206] Beginning negotiation
0806 18:34:45.252110 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 18:34:45.252363 (+ 253us) server_negotiation.cc:407] Connection header received
0806 18:34:45.252589 (+ 226us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:34:45.252592 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:34:45.252644 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:34:45.252730 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:34:45.253419 (+ 689us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:34:45.253936 (+ 517us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:34:45.254793 (+ 857us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:34:45.255023 (+ 230us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:34:45.255134 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:34:45.258416 (+ 3282us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:46812: BlockingRecv error: recv got EOF from 127.0.0.1:46812 (error 108)
Metrics: {"server-negotiator.queue_time_us":48}
Aug 06 18:34:45 dist-test-slave-1xrh krb5kdc[27989](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041285, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.19.194.62@KRBTEST.COM
I20260806 18:34:45.272735 28085 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:34:45.262479 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46824 (local address 127.19.194.62:33607)
0806 18:34:45.262573 (+ 94us) server_negotiation.cc:206] Beginning negotiation
0806 18:34:45.262577 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 18:34:45.263062 (+ 485us) server_negotiation.cc:407] Connection header received
0806 18:34:45.263235 (+ 173us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:34:45.263238 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:34:45.263277 (+ 39us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:34:45.263328 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:34:45.264098 (+ 770us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:34:45.264890 (+ 792us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:34:45.265991 (+ 1101us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:34:45.266288 (+ 297us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:34:45.266373 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:34:45.269271 (+ 2898us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:34:45.269275 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:34:45.269277 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 18:34:45.269310 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:34:45.270915 (+ 1605us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:34:45.271447 (+ 532us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:34:45.271451 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:34:45.271453 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:34:45.271531 (+ 78us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:34:45.271778 (+ 247us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:34:45.271781 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:34:45.271782 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:34:45.271994 (+ 212us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:34:45.272110 (+ 116us) server_negotiation.cc:1035] Waiting for connection context
0806 18:34:45.272454 (+ 344us) server_negotiation.cc:299] Negotiation successful
0806 18:34:45.272561 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260806 18:34:45.275885 28035 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:46824:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260806 18:34:45.276338 28035 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 18:34:45.281731 28173 tablet_service.cc:1467] Processing CreateTablet for tablet de44e30581134455b96443c854791936 (DEFAULT_TABLE table=test-table [id=1929fb0b0cc241d6b798af3cb57545c6]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 18:34:45.281886 28313 tablet_service.cc:1467] Processing CreateTablet for tablet de44e30581134455b96443c854791936 (DEFAULT_TABLE table=test-table [id=1929fb0b0cc241d6b798af3cb57545c6]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 18:34:45.282058 28173 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet de44e30581134455b96443c854791936. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 18:34:45.282145 28313 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet de44e30581134455b96443c854791936. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 18:34:45.284581 28559 tablet_bootstrap.cc:492] T de44e30581134455b96443c854791936 P 4575246688ab47bdb5abf0da5c702767: Bootstrap starting.
I20260806 18:34:45.285636 28559 tablet_bootstrap.cc:654] T de44e30581134455b96443c854791936 P 4575246688ab47bdb5abf0da5c702767: Neither blocks nor log segments found. Creating new log.
I20260806 18:34:45.286559 28562 tablet_bootstrap.cc:492] T de44e30581134455b96443c854791936 P 885703862b27424890515f701c83aa68: Bootstrap starting.
I20260806 18:34:45.286620 28462 tablet_service.cc:1467] Processing CreateTablet for tablet de44e30581134455b96443c854791936 (DEFAULT_TABLE table=test-table [id=1929fb0b0cc241d6b798af3cb57545c6]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 18:34:45.286909 28462 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet de44e30581134455b96443c854791936. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 18:34:45.286999 28559 tablet_bootstrap.cc:492] T de44e30581134455b96443c854791936 P 4575246688ab47bdb5abf0da5c702767: No bootstrap required, opened a new log
I20260806 18:34:45.287077 28559 ts_tablet_manager.cc:1397] T de44e30581134455b96443c854791936 P 4575246688ab47bdb5abf0da5c702767: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260806 18:34:45.287485 28562 tablet_bootstrap.cc:654] T de44e30581134455b96443c854791936 P 885703862b27424890515f701c83aa68: Neither blocks nor log segments found. Creating new log.
I20260806 18:34:45.287603 28559 raft_consensus.cc:348] T de44e30581134455b96443c854791936 P 4575246688ab47bdb5abf0da5c702767 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "885703862b27424890515f701c83aa68" member_type: VOTER last_known_addr { host: "127.19.194.1" port: 34859 } } peers { permanent_uuid: "4575246688ab47bdb5abf0da5c702767" member_type: VOTER last_known_addr { host: "127.19.194.2" port: 44765 } } peers { permanent_uuid: "63955300c20044878851d5e44e44155c" member_type: VOTER last_known_addr { host: "127.19.194.3" port: 46307 } }
I20260806 18:34:45.287710 28559 raft_consensus.cc:374] T de44e30581134455b96443c854791936 P 4575246688ab47bdb5abf0da5c702767 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 18:34:45.287745 28559 raft_consensus.cc:729] T de44e30581134455b96443c854791936 P 4575246688ab47bdb5abf0da5c702767 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4575246688ab47bdb5abf0da5c702767, State: Initialized, Role: FOLLOWER
I20260806 18:34:45.287883 28559 consensus_queue.cc:260] T de44e30581134455b96443c854791936 P 4575246688ab47bdb5abf0da5c702767 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "885703862b27424890515f701c83aa68" member_type: VOTER last_known_addr { host: "127.19.194.1" port: 34859 } } peers { permanent_uuid: "4575246688ab47bdb5abf0da5c702767" member_type: VOTER last_known_addr { host: "127.19.194.2" port: 44765 } } peers { permanent_uuid: "63955300c20044878851d5e44e44155c" member_type: VOTER last_known_addr { host: "127.19.194.3" port: 46307 } }
I20260806 18:34:45.288162 28559 ts_tablet_manager.cc:1428] T de44e30581134455b96443c854791936 P 4575246688ab47bdb5abf0da5c702767: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260806 18:34:45.289238 28560 tablet_bootstrap.cc:492] T de44e30581134455b96443c854791936 P 63955300c20044878851d5e44e44155c: Bootstrap starting.
I20260806 18:34:45.290241 28560 tablet_bootstrap.cc:654] T de44e30581134455b96443c854791936 P 63955300c20044878851d5e44e44155c: Neither blocks nor log segments found. Creating new log.
I20260806 18:34:45.293466 28562 tablet_bootstrap.cc:492] T de44e30581134455b96443c854791936 P 885703862b27424890515f701c83aa68: No bootstrap required, opened a new log
I20260806 18:34:45.293562 28562 ts_tablet_manager.cc:1397] T de44e30581134455b96443c854791936 P 885703862b27424890515f701c83aa68: Time spent bootstrapping tablet: real 0.007s user 0.002s sys 0.000s
I20260806 18:34:45.293805 28560 tablet_bootstrap.cc:492] T de44e30581134455b96443c854791936 P 63955300c20044878851d5e44e44155c: No bootstrap required, opened a new log
I20260806 18:34:45.293889 28560 ts_tablet_manager.cc:1397] T de44e30581134455b96443c854791936 P 63955300c20044878851d5e44e44155c: Time spent bootstrapping tablet: real 0.005s user 0.001s sys 0.000s
I20260806 18:34:45.294052 28562 raft_consensus.cc:348] T de44e30581134455b96443c854791936 P 885703862b27424890515f701c83aa68 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "885703862b27424890515f701c83aa68" member_type: VOTER last_known_addr { host: "127.19.194.1" port: 34859 } } peers { permanent_uuid: "4575246688ab47bdb5abf0da5c702767" member_type: VOTER last_known_addr { host: "127.19.194.2" port: 44765 } } peers { permanent_uuid: "63955300c20044878851d5e44e44155c" member_type: VOTER last_known_addr { host: "127.19.194.3" port: 46307 } }
I20260806 18:34:45.294183 28562 raft_consensus.cc:374] T de44e30581134455b96443c854791936 P 885703862b27424890515f701c83aa68 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 18:34:45.294226 28562 raft_consensus.cc:729] T de44e30581134455b96443c854791936 P 885703862b27424890515f701c83aa68 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 885703862b27424890515f701c83aa68, State: Initialized, Role: FOLLOWER
I20260806 18:34:45.294306 28560 raft_consensus.cc:348] T de44e30581134455b96443c854791936 P 63955300c20044878851d5e44e44155c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "885703862b27424890515f701c83aa68" member_type: VOTER last_known_addr { host: "127.19.194.1" port: 34859 } } peers { permanent_uuid: "4575246688ab47bdb5abf0da5c702767" member_type: VOTER last_known_addr { host: "127.19.194.2" port: 44765 } } peers { permanent_uuid: "63955300c20044878851d5e44e44155c" member_type: VOTER last_known_addr { host: "127.19.194.3" port: 46307 } }
I20260806 18:34:45.294420 28560 raft_consensus.cc:374] T de44e30581134455b96443c854791936 P 63955300c20044878851d5e44e44155c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 18:34:45.294448 28560 raft_consensus.cc:729] T de44e30581134455b96443c854791936 P 63955300c20044878851d5e44e44155c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 63955300c20044878851d5e44e44155c, State: Initialized, Role: FOLLOWER
I20260806 18:34:45.294374 28562 consensus_queue.cc:260] T de44e30581134455b96443c854791936 P 885703862b27424890515f701c83aa68 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "885703862b27424890515f701c83aa68" member_type: VOTER last_known_addr { host: "127.19.194.1" port: 34859 } } peers { permanent_uuid: "4575246688ab47bdb5abf0da5c702767" member_type: VOTER last_known_addr { host: "127.19.194.2" port: 44765 } } peers { permanent_uuid: "63955300c20044878851d5e44e44155c" member_type: VOTER last_known_addr { host: "127.19.194.3" port: 46307 } }
I20260806 18:34:45.294564 28560 consensus_queue.cc:260] T de44e30581134455b96443c854791936 P 63955300c20044878851d5e44e44155c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "885703862b27424890515f701c83aa68" member_type: VOTER last_known_addr { host: "127.19.194.1" port: 34859 } } peers { permanent_uuid: "4575246688ab47bdb5abf0da5c702767" member_type: VOTER last_known_addr { host: "127.19.194.2" port: 44765 } } peers { permanent_uuid: "63955300c20044878851d5e44e44155c" member_type: VOTER last_known_addr { host: "127.19.194.3" port: 46307 } }
I20260806 18:34:45.294680 28562 ts_tablet_manager.cc:1428] T de44e30581134455b96443c854791936 P 885703862b27424890515f701c83aa68: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260806 18:34:45.294801 28560 ts_tablet_manager.cc:1428] T de44e30581134455b96443c854791936 P 63955300c20044878851d5e44e44155c: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260806 18:34:45.348359 28567 raft_consensus.cc:482] T de44e30581134455b96443c854791936 P 885703862b27424890515f701c83aa68 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 18:34:45.348505 28567 raft_consensus.cc:504] T de44e30581134455b96443c854791936 P 885703862b27424890515f701c83aa68 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "885703862b27424890515f701c83aa68" member_type: VOTER last_known_addr { host: "127.19.194.1" port: 34859 } } peers { permanent_uuid: "4575246688ab47bdb5abf0da5c702767" member_type: VOTER last_known_addr { host: "127.19.194.2" port: 44765 } } peers { permanent_uuid: "63955300c20044878851d5e44e44155c" member_type: VOTER last_known_addr { host: "127.19.194.3" port: 46307 } }
I20260806 18:34:45.348986 28567 leader_election.cc:288] T de44e30581134455b96443c854791936 P 885703862b27424890515f701c83aa68 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4575246688ab47bdb5abf0da5c702767 (127.19.194.2:44765), 63955300c20044878851d5e44e44155c (127.19.194.3:46307)
I20260806 18:34:45.349408 28334 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "de44e30581134455b96443c854791936" candidate_uuid: "885703862b27424890515f701c83aa68" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4575246688ab47bdb5abf0da5c702767" is_pre_election: true
I20260806 18:34:45.349485 28465 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "de44e30581134455b96443c854791936" candidate_uuid: "885703862b27424890515f701c83aa68" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "63955300c20044878851d5e44e44155c" is_pre_election: true
I20260806 18:34:45.349570 28334 raft_consensus.cc:2454] T de44e30581134455b96443c854791936 P 4575246688ab47bdb5abf0da5c702767 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 885703862b27424890515f701c83aa68 in term 0.
I20260806 18:34:45.349659 28465 raft_consensus.cc:2454] T de44e30581134455b96443c854791936 P 63955300c20044878851d5e44e44155c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 885703862b27424890515f701c83aa68 in term 0.
I20260806 18:34:45.349903 28126 leader_election.cc:302] T de44e30581134455b96443c854791936 P 885703862b27424890515f701c83aa68 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4575246688ab47bdb5abf0da5c702767, 885703862b27424890515f701c83aa68; no voters:
I20260806 18:34:45.350174 28567 raft_consensus.cc:2781] T de44e30581134455b96443c854791936 P 885703862b27424890515f701c83aa68 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 18:34:45.350244 28567 raft_consensus.cc:482] T de44e30581134455b96443c854791936 P 885703862b27424890515f701c83aa68 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 18:34:45.350297 28567 raft_consensus.cc:3037] T de44e30581134455b96443c854791936 P 885703862b27424890515f701c83aa68 [term 0 FOLLOWER]: Advancing to term 1
I20260806 18:34:45.351634 28567 raft_consensus.cc:504] T de44e30581134455b96443c854791936 P 885703862b27424890515f701c83aa68 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "885703862b27424890515f701c83aa68" member_type: VOTER last_known_addr { host: "127.19.194.1" port: 34859 } } peers { permanent_uuid: "4575246688ab47bdb5abf0da5c702767" member_type: VOTER last_known_addr { host: "127.19.194.2" port: 44765 } } peers { permanent_uuid: "63955300c20044878851d5e44e44155c" member_type: VOTER last_known_addr { host: "127.19.194.3" port: 46307 } }
I20260806 18:34:45.351990 28567 leader_election.cc:288] T de44e30581134455b96443c854791936 P 885703862b27424890515f701c83aa68 [CANDIDATE]: Term 1 election: Requested vote from peers 4575246688ab47bdb5abf0da5c702767 (127.19.194.2:44765), 63955300c20044878851d5e44e44155c (127.19.194.3:46307)
I20260806 18:34:45.352311 28334 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "de44e30581134455b96443c854791936" candidate_uuid: "885703862b27424890515f701c83aa68" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4575246688ab47bdb5abf0da5c702767"
I20260806 18:34:45.352447 28334 raft_consensus.cc:3037] T de44e30581134455b96443c854791936 P 4575246688ab47bdb5abf0da5c702767 [term 0 FOLLOWER]: Advancing to term 1
I20260806 18:34:45.352497 28465 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "de44e30581134455b96443c854791936" candidate_uuid: "885703862b27424890515f701c83aa68" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "63955300c20044878851d5e44e44155c"
I20260806 18:34:45.352631 28465 raft_consensus.cc:3037] T de44e30581134455b96443c854791936 P 63955300c20044878851d5e44e44155c [term 0 FOLLOWER]: Advancing to term 1
I20260806 18:34:45.353652 28334 raft_consensus.cc:2454] T de44e30581134455b96443c854791936 P 4575246688ab47bdb5abf0da5c702767 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 885703862b27424890515f701c83aa68 in term 1.
I20260806 18:34:45.353915 28465 raft_consensus.cc:2454] T de44e30581134455b96443c854791936 P 63955300c20044878851d5e44e44155c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 885703862b27424890515f701c83aa68 in term 1.
I20260806 18:34:45.353942 28126 leader_election.cc:302] T de44e30581134455b96443c854791936 P 885703862b27424890515f701c83aa68 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4575246688ab47bdb5abf0da5c702767, 885703862b27424890515f701c83aa68; no voters:
I20260806 18:34:45.354112 28567 raft_consensus.cc:2781] T de44e30581134455b96443c854791936 P 885703862b27424890515f701c83aa68 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 18:34:45.354225 28567 raft_consensus.cc:686] T de44e30581134455b96443c854791936 P 885703862b27424890515f701c83aa68 [term 1 LEADER]: Becoming Leader. State: Replica: 885703862b27424890515f701c83aa68, State: Running, Role: LEADER
I20260806 18:34:45.354372 28567 consensus_queue.cc:237] T de44e30581134455b96443c854791936 P 885703862b27424890515f701c83aa68 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "885703862b27424890515f701c83aa68" member_type: VOTER last_known_addr { host: "127.19.194.1" port: 34859 } } peers { permanent_uuid: "4575246688ab47bdb5abf0da5c702767" member_type: VOTER last_known_addr { host: "127.19.194.2" port: 44765 } } peers { permanent_uuid: "63955300c20044878851d5e44e44155c" member_type: VOTER last_known_addr { host: "127.19.194.3" port: 46307 } }
I20260806 18:34:45.356318 28035 catalog_manager.cc:5495] T de44e30581134455b96443c854791936 P 885703862b27424890515f701c83aa68 reported cstate change: term changed from 0 to 1, leader changed from <none> to 885703862b27424890515f701c83aa68 (127.19.194.1). New cstate: current_term: 1 leader_uuid: "885703862b27424890515f701c83aa68" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "885703862b27424890515f701c83aa68" member_type: VOTER last_known_addr { host: "127.19.194.1" port: 34859 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4575246688ab47bdb5abf0da5c702767" member_type: VOTER last_known_addr { host: "127.19.194.2" port: 44765 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "63955300c20044878851d5e44e44155c" member_type: VOTER last_known_addr { host: "127.19.194.3" port: 46307 } health_report { overall_health: UNKNOWN } } }
I20260806 18:34:45.374197 28558 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:34:45.370767 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33856 (local address 127.19.194.1:34859)
0806 18:34:45.370927 (+ 160us) server_negotiation.cc:206] Beginning negotiation
0806 18:34:45.370932 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 18:34:45.370949 (+ 17us) server_negotiation.cc:407] Connection header received
0806 18:34:45.371032 (+ 83us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:34:45.371035 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:34:45.371089 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:34:45.371200 (+ 111us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 18:34:45.371466 (+ 266us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:34:45.372222 (+ 756us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:34:45.373046 (+ 824us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:34:45.373387 (+ 341us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:34:45.373499 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:34:45.373729 (+ 230us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 18:34:45.373821 (+ 92us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 18:34:45.373902 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0806 18:34:45.374008 (+ 106us) server_negotiation.cc:299] Negotiation successful
0806 18:34:45.374063 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260806 18:34:45.376251 28465 raft_consensus.cc:1261] T de44e30581134455b96443c854791936 P 63955300c20044878851d5e44e44155c [term 1 FOLLOWER]: Refusing update from remote peer 885703862b27424890515f701c83aa68: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 18:34:45.376329 28334 raft_consensus.cc:1261] T de44e30581134455b96443c854791936 P 4575246688ab47bdb5abf0da5c702767 [term 1 FOLLOWER]: Refusing update from remote peer 885703862b27424890515f701c83aa68: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 18:34:45.376749 28567 consensus_queue.cc:1035] T de44e30581134455b96443c854791936 P 885703862b27424890515f701c83aa68 [LEADER]: Connected to new peer: Peer: permanent_uuid: "63955300c20044878851d5e44e44155c" member_type: VOTER last_known_addr { host: "127.19.194.3" port: 46307 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 18:34:45.376965 28578 consensus_queue.cc:1035] T de44e30581134455b96443c854791936 P 885703862b27424890515f701c83aa68 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4575246688ab47bdb5abf0da5c702767" member_type: VOTER last_known_addr { host: "127.19.194.2" port: 44765 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 18:34:45.385267 28035 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:46824:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 18:34:45.385404 28035 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:46824:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 18:34:45.387745 28035 catalog_manager.cc:5782] T 00000000000000000000000000000000 P d0282f2efb864823bbf90a21992ab39d: Sending DeleteTablet for 3 replicas of tablet de44e30581134455b96443c854791936
I20260806 18:34:45.388298 28313 tablet_service.cc:1514] Processing DeleteTablet for tablet de44e30581134455b96443c854791936 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 18:34:45 UTC) from {username='oryx', principal='oryx/127.19.194.62@KRBTEST.COM'} at 127.0.0.1:51842
I20260806 18:34:45.388494 28462 tablet_service.cc:1514] Processing DeleteTablet for tablet de44e30581134455b96443c854791936 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 18:34:45 UTC) from {username='oryx', principal='oryx/127.19.194.62@KRBTEST.COM'} at 127.0.0.1:41986
I20260806 18:34:45.388545 28461 tablet_service.cc:1514] Processing DeleteTablet for tablet de44e30581134455b96443c854791936 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 18:34:45 UTC) from {username='oryx', principal='oryx/127.19.194.62@KRBTEST.COM'} at 127.0.0.1:41986
I20260806 18:34:45.388697 28584 tablet_replica.cc:330] T de44e30581134455b96443c854791936 P 4575246688ab47bdb5abf0da5c702767: stopping tablet replica
I20260806 18:34:45.388841 28584 raft_consensus.cc:2229] T de44e30581134455b96443c854791936 P 4575246688ab47bdb5abf0da5c702767 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 18:34:45.388988 28584 raft_consensus.cc:2258] T de44e30581134455b96443c854791936 P 4575246688ab47bdb5abf0da5c702767 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 18:34:45.389179 28583 tablet_replica.cc:330] T de44e30581134455b96443c854791936 P 63955300c20044878851d5e44e44155c: stopping tablet replica
I20260806 18:34:45.389103 28035 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:46824:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260806 18:34:45.389283 28583 raft_consensus.cc:2229] T de44e30581134455b96443c854791936 P 63955300c20044878851d5e44e44155c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 18:34:45.389392 28583 raft_consensus.cc:2258] T de44e30581134455b96443c854791936 P 63955300c20044878851d5e44e44155c [term 1 FOLLOWER]: Raft consensus is shut down!
W20260806 18:34:45.389536 28035 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 18:34:45.389904 28173 tablet_service.cc:1514] Processing DeleteTablet for tablet de44e30581134455b96443c854791936 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 18:34:45 UTC) from {username='oryx', principal='oryx/127.19.194.62@KRBTEST.COM'} at 127.0.0.1:33838
I20260806 18:34:45.390054 28585 tablet_replica.cc:330] T de44e30581134455b96443c854791936 P 885703862b27424890515f701c83aa68: stopping tablet replica
I20260806 18:34:45.390146 28585 raft_consensus.cc:2229] T de44e30581134455b96443c854791936 P 885703862b27424890515f701c83aa68 [term 1 LEADER]: Raft consensus shutting down.
I20260806 18:34:45.390291 28585 raft_consensus.cc:2258] T de44e30581134455b96443c854791936 P 885703862b27424890515f701c83aa68 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 18:34:45.390867 28583 ts_tablet_manager.cc:1905] T de44e30581134455b96443c854791936 P 63955300c20044878851d5e44e44155c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 18:34:45.390966 28585 ts_tablet_manager.cc:1905] T de44e30581134455b96443c854791936 P 885703862b27424890515f701c83aa68: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 18:34:45.392915 28583 ts_tablet_manager.cc:1918] T de44e30581134455b96443c854791936 P 63955300c20044878851d5e44e44155c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 18:34:45.393023 28583 log.cc:1196] T de44e30581134455b96443c854791936 P 63955300c20044878851d5e44e44155c: Deleting WAL directory at /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786041246094603-20232-0/minicluster-data/ts-2/wal/wals/de44e30581134455b96443c854791936
I20260806 18:34:45.393277 28583 ts_tablet_manager.cc:1939] T de44e30581134455b96443c854791936 P 63955300c20044878851d5e44e44155c: Deleting consensus metadata
I20260806 18:34:45.393548 28585 ts_tablet_manager.cc:1918] T de44e30581134455b96443c854791936 P 885703862b27424890515f701c83aa68: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 18:34:45.393620 28585 log.cc:1196] T de44e30581134455b96443c854791936 P 885703862b27424890515f701c83aa68: Deleting WAL directory at /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786041246094603-20232-0/minicluster-data/ts-0/wal/wals/de44e30581134455b96443c854791936
I20260806 18:34:45.393882 28585 ts_tablet_manager.cc:1939] T de44e30581134455b96443c854791936 P 885703862b27424890515f701c83aa68: Deleting consensus metadata
I20260806 18:34:45.394133 28019 catalog_manager.cc:4841] TS 63955300c20044878851d5e44e44155c (127.19.194.3:46307): tablet de44e30581134455b96443c854791936 (table test-table [id=1929fb0b0cc241d6b798af3cb57545c6]) successfully deleted
I20260806 18:34:45.394521 28022 catalog_manager.cc:4841] TS 885703862b27424890515f701c83aa68 (127.19.194.1:34859): tablet de44e30581134455b96443c854791936 (table test-table [id=1929fb0b0cc241d6b798af3cb57545c6]) successfully deleted
W20260806 18:34:45.394976 28019 catalog_manager.cc:4808] TS 63955300c20044878851d5e44e44155c (127.19.194.3:46307): delete failed for tablet de44e30581134455b96443c854791936 because the tablet was not found. No further retry: Not found: Tablet not found: de44e30581134455b96443c854791936
I20260806 18:34:45.395290 28584 ts_tablet_manager.cc:1905] T de44e30581134455b96443c854791936 P 4575246688ab47bdb5abf0da5c702767: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 18:34:45.397444 28173 tablet_service.cc:1467] Processing CreateTablet for tablet 0a679fe952314ff8b936ca074c980c93 (DEFAULT_TABLE table=test-table [id=856bb2f982d148748b19b894c6d127da]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 18:34:45.397562 28313 tablet_service.cc:1467] Processing CreateTablet for tablet 0a679fe952314ff8b936ca074c980c93 (DEFAULT_TABLE table=test-table [id=856bb2f982d148748b19b894c6d127da]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 18:34:45.397677 28173 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0a679fe952314ff8b936ca074c980c93. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 18:34:45.397848 28584 ts_tablet_manager.cc:1918] T de44e30581134455b96443c854791936 P 4575246688ab47bdb5abf0da5c702767: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 18:34:45.397882 28313 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0a679fe952314ff8b936ca074c980c93. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 18:34:45.397915 28584 log.cc:1196] T de44e30581134455b96443c854791936 P 4575246688ab47bdb5abf0da5c702767: Deleting WAL directory at /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786041246094603-20232-0/minicluster-data/ts-1/wal/wals/de44e30581134455b96443c854791936
I20260806 18:34:45.398173 28584 ts_tablet_manager.cc:1939] T de44e30581134455b96443c854791936 P 4575246688ab47bdb5abf0da5c702767: Deleting consensus metadata
I20260806 18:34:45.399258 28021 catalog_manager.cc:4841] TS 4575246688ab47bdb5abf0da5c702767 (127.19.194.2:44765): tablet de44e30581134455b96443c854791936 (table test-table [id=1929fb0b0cc241d6b798af3cb57545c6]) successfully deleted
I20260806 18:34:45.399787 28461 tablet_service.cc:1467] Processing CreateTablet for tablet 0a679fe952314ff8b936ca074c980c93 (DEFAULT_TABLE table=test-table [id=856bb2f982d148748b19b894c6d127da]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 18:34:45.400043 28461 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0a679fe952314ff8b936ca074c980c93. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 18:34:45.400208 28559 tablet_bootstrap.cc:492] T 0a679fe952314ff8b936ca074c980c93 P 4575246688ab47bdb5abf0da5c702767: Bootstrap starting.
I20260806 18:34:45.400874 28559 tablet_bootstrap.cc:654] T 0a679fe952314ff8b936ca074c980c93 P 4575246688ab47bdb5abf0da5c702767: Neither blocks nor log segments found. Creating new log.
I20260806 18:34:45.402127 28559 tablet_bootstrap.cc:492] T 0a679fe952314ff8b936ca074c980c93 P 4575246688ab47bdb5abf0da5c702767: No bootstrap required, opened a new log
I20260806 18:34:45.402215 28559 ts_tablet_manager.cc:1397] T 0a679fe952314ff8b936ca074c980c93 P 4575246688ab47bdb5abf0da5c702767: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260806 18:34:45.402647 28560 tablet_bootstrap.cc:492] T 0a679fe952314ff8b936ca074c980c93 P 63955300c20044878851d5e44e44155c: Bootstrap starting.
I20260806 18:34:45.402653 28559 raft_consensus.cc:348] T 0a679fe952314ff8b936ca074c980c93 P 4575246688ab47bdb5abf0da5c702767 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "885703862b27424890515f701c83aa68" member_type: VOTER last_known_addr { host: "127.19.194.1" port: 34859 } } peers { permanent_uuid: "4575246688ab47bdb5abf0da5c702767" member_type: VOTER last_known_addr { host: "127.19.194.2" port: 44765 } } peers { permanent_uuid: "63955300c20044878851d5e44e44155c" member_type: VOTER last_known_addr { host: "127.19.194.3" port: 46307 } }
I20260806 18:34:45.402788 28559 raft_consensus.cc:374] T 0a679fe952314ff8b936ca074c980c93 P 4575246688ab47bdb5abf0da5c702767 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 18:34:45.402832 28559 raft_consensus.cc:729] T 0a679fe952314ff8b936ca074c980c93 P 4575246688ab47bdb5abf0da5c702767 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4575246688ab47bdb5abf0da5c702767, State: Initialized, Role: FOLLOWER
I20260806 18:34:45.402968 28559 consensus_queue.cc:260] T 0a679fe952314ff8b936ca074c980c93 P 4575246688ab47bdb5abf0da5c702767 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "885703862b27424890515f701c83aa68" member_type: VOTER last_known_addr { host: "127.19.194.1" port: 34859 } } peers { permanent_uuid: "4575246688ab47bdb5abf0da5c702767" member_type: VOTER last_known_addr { host: "127.19.194.2" port: 44765 } } peers { permanent_uuid: "63955300c20044878851d5e44e44155c" member_type: VOTER last_known_addr { host: "127.19.194.3" port: 46307 } }
I20260806 18:34:45.403255 28559 ts_tablet_manager.cc:1428] T 0a679fe952314ff8b936ca074c980c93 P 4575246688ab47bdb5abf0da5c702767: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260806 18:34:45.403623 28560 tablet_bootstrap.cc:654] T 0a679fe952314ff8b936ca074c980c93 P 63955300c20044878851d5e44e44155c: Neither blocks nor log segments found. Creating new log.
I20260806 18:34:45.404889 28560 tablet_bootstrap.cc:492] T 0a679fe952314ff8b936ca074c980c93 P 63955300c20044878851d5e44e44155c: No bootstrap required, opened a new log
I20260806 18:34:45.404990 28560 ts_tablet_manager.cc:1397] T 0a679fe952314ff8b936ca074c980c93 P 63955300c20044878851d5e44e44155c: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.002s
I20260806 18:34:45.405454 28560 raft_consensus.cc:348] T 0a679fe952314ff8b936ca074c980c93 P 63955300c20044878851d5e44e44155c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "885703862b27424890515f701c83aa68" member_type: VOTER last_known_addr { host: "127.19.194.1" port: 34859 } } peers { permanent_uuid: "4575246688ab47bdb5abf0da5c702767" member_type: VOTER last_known_addr { host: "127.19.194.2" port: 44765 } } peers { permanent_uuid: "63955300c20044878851d5e44e44155c" member_type: VOTER last_known_addr { host: "127.19.194.3" port: 46307 } }
I20260806 18:34:45.405581 28560 raft_consensus.cc:374] T 0a679fe952314ff8b936ca074c980c93 P 63955300c20044878851d5e44e44155c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 18:34:45.405623 28560 raft_consensus.cc:729] T 0a679fe952314ff8b936ca074c980c93 P 63955300c20044878851d5e44e44155c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 63955300c20044878851d5e44e44155c, State: Initialized, Role: FOLLOWER
I20260806 18:34:45.405736 28560 consensus_queue.cc:260] T 0a679fe952314ff8b936ca074c980c93 P 63955300c20044878851d5e44e44155c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "885703862b27424890515f701c83aa68" member_type: VOTER last_known_addr { host: "127.19.194.1" port: 34859 } } peers { permanent_uuid: "4575246688ab47bdb5abf0da5c702767" member_type: VOTER last_known_addr { host: "127.19.194.2" port: 44765 } } peers { permanent_uuid: "63955300c20044878851d5e44e44155c" member_type: VOTER last_known_addr { host: "127.19.194.3" port: 46307 } }
I20260806 18:34:45.405992 28560 ts_tablet_manager.cc:1428] T 0a679fe952314ff8b936ca074c980c93 P 63955300c20044878851d5e44e44155c: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20260806 18:34:45.406785 28562 tablet_bootstrap.cc:492] T 0a679fe952314ff8b936ca074c980c93 P 885703862b27424890515f701c83aa68: Bootstrap starting.
I20260806 18:34:45.407630 28562 tablet_bootstrap.cc:654] T 0a679fe952314ff8b936ca074c980c93 P 885703862b27424890515f701c83aa68: Neither blocks nor log segments found. Creating new log.
I20260806 18:34:45.409106 28562 tablet_bootstrap.cc:492] T 0a679fe952314ff8b936ca074c980c93 P 885703862b27424890515f701c83aa68: No bootstrap required, opened a new log
I20260806 18:34:45.409215 28562 ts_tablet_manager.cc:1397] T 0a679fe952314ff8b936ca074c980c93 P 885703862b27424890515f701c83aa68: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260806 18:34:45.409571 28562 raft_consensus.cc:348] T 0a679fe952314ff8b936ca074c980c93 P 885703862b27424890515f701c83aa68 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "885703862b27424890515f701c83aa68" member_type: VOTER last_known_addr { host: "127.19.194.1" port: 34859 } } peers { permanent_uuid: "4575246688ab47bdb5abf0da5c702767" member_type: VOTER last_known_addr { host: "127.19.194.2" port: 44765 } } peers { permanent_uuid: "63955300c20044878851d5e44e44155c" member_type: VOTER last_known_addr { host: "127.19.194.3" port: 46307 } }
I20260806 18:34:45.409686 28562 raft_consensus.cc:374] T 0a679fe952314ff8b936ca074c980c93 P 885703862b27424890515f701c83aa68 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 18:34:45.409730 28562 raft_consensus.cc:729] T 0a679fe952314ff8b936ca074c980c93 P 885703862b27424890515f701c83aa68 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 885703862b27424890515f701c83aa68, State: Initialized, Role: FOLLOWER
I20260806 18:34:45.409855 28562 consensus_queue.cc:260] T 0a679fe952314ff8b936ca074c980c93 P 885703862b27424890515f701c83aa68 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "885703862b27424890515f701c83aa68" member_type: VOTER last_known_addr { host: "127.19.194.1" port: 34859 } } peers { permanent_uuid: "4575246688ab47bdb5abf0da5c702767" member_type: VOTER last_known_addr { host: "127.19.194.2" port: 44765 } } peers { permanent_uuid: "63955300c20044878851d5e44e44155c" member_type: VOTER last_known_addr { host: "127.19.194.3" port: 46307 } }
I20260806 18:34:45.410135 28562 ts_tablet_manager.cc:1428] T 0a679fe952314ff8b936ca074c980c93 P 885703862b27424890515f701c83aa68: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
W20260806 18:34:45.445883 28532 tablet.cc:2369] T 0a679fe952314ff8b936ca074c980c93 P 63955300c20044878851d5e44e44155c: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 18:34:45.677019 28565 raft_consensus.cc:482] T 0a679fe952314ff8b936ca074c980c93 P 63955300c20044878851d5e44e44155c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 18:34:45.677162 28565 raft_consensus.cc:504] T 0a679fe952314ff8b936ca074c980c93 P 63955300c20044878851d5e44e44155c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "885703862b27424890515f701c83aa68" member_type: VOTER last_known_addr { host: "127.19.194.1" port: 34859 } } peers { permanent_uuid: "4575246688ab47bdb5abf0da5c702767" member_type: VOTER last_known_addr { host: "127.19.194.2" port: 44765 } } peers { permanent_uuid: "63955300c20044878851d5e44e44155c" member_type: VOTER last_known_addr { host: "127.19.194.3" port: 46307 } }
I20260806 18:34:45.678234 28565 leader_election.cc:288] T 0a679fe952314ff8b936ca074c980c93 P 63955300c20044878851d5e44e44155c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 885703862b27424890515f701c83aa68 (127.19.194.1:34859), 4575246688ab47bdb5abf0da5c702767 (127.19.194.2:44765)
I20260806 18:34:45.682552 28558 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:34:45.678553 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.194.3:37877 (local address 127.19.194.1:34859)
0806 18:34:45.678688 (+ 135us) server_negotiation.cc:206] Beginning negotiation
0806 18:34:45.678693 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 18:34:45.678888 (+ 195us) server_negotiation.cc:407] Connection header received
0806 18:34:45.679075 (+ 187us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:34:45.679079 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:34:45.679181 (+ 102us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:34:45.679290 (+ 109us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 18:34:45.679650 (+ 360us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:34:45.680126 (+ 476us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:34:45.681370 (+ 1244us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:34:45.681911 (+ 541us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:34:45.682008 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:34:45.682075 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0806 18:34:45.682328 (+ 253us) server_negotiation.cc:299] Negotiation successful
0806 18:34:45.682421 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260806 18:34:45.682554 28646 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:34:45.678448 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.194.1:34859 (local address 127.19.194.3:37877)
0806 18:34:45.678757 (+ 309us) negotiation.cc:107] Waiting for socket to connect
0806 18:34:45.678774 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0806 18:34:45.678952 (+ 178us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:34:45.679322 (+ 370us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:34:45.679326 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:34:45.679351 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 18:34:45.679499 (+ 148us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:34:45.679504 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:34:45.680298 (+ 794us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:34:45.680304 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:34:45.681247 (+ 943us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:34:45.681257 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:34:45.682028 (+ 771us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:34:45.682034 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:34:45.682141 (+ 107us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:34:45.682169 (+ 28us) client_negotiation.cc:769] Sending connection context
0806 18:34:45.682280 (+ 111us) client_negotiation.cc:240] Negotiation successful
0806 18:34:45.682341 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":242,"mutex_wait_us":4,"spinlock_wait_cycles":34176,"thread_start_us":75,"threads_started":1}
I20260806 18:34:45.682646 28557 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:34:45.678589 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.194.3:46111 (local address 127.19.194.2:44765)
0806 18:34:45.678750 (+ 161us) server_negotiation.cc:206] Beginning negotiation
0806 18:34:45.678754 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 18:34:45.678879 (+ 125us) server_negotiation.cc:407] Connection header received
0806 18:34:45.679109 (+ 230us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:34:45.679111 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:34:45.679175 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:34:45.679235 (+ 60us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 18:34:45.679579 (+ 344us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:34:45.680107 (+ 528us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:34:45.681419 (+ 1312us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:34:45.681992 (+ 573us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:34:45.682122 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:34:45.682194 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0806 18:34:45.682443 (+ 249us) server_negotiation.cc:299] Negotiation successful
0806 18:34:45.682514 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":77}
I20260806 18:34:45.682613 28647 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:34:45.678471 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.194.2:44765 (local address 127.19.194.3:46111)
0806 18:34:45.678783 (+ 312us) negotiation.cc:107] Waiting for socket to connect
0806 18:34:45.678795 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0806 18:34:45.678969 (+ 174us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:34:45.679284 (+ 315us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:34:45.679287 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:34:45.679304 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 18:34:45.679462 (+ 158us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:34:45.679469 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:34:45.680243 (+ 774us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:34:45.680246 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:34:45.681256 (+ 1010us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:34:45.681265 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:34:45.682169 (+ 904us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:34:45.682174 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:34:45.682302 (+ 128us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:34:45.682327 (+ 25us) client_negotiation.cc:769] Sending connection context
0806 18:34:45.682408 (+ 81us) client_negotiation.cc:240] Negotiation successful
0806 18:34:45.682473 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":248,"mutex_wait_us":75,"spinlock_wait_cycles":18048,"thread_start_us":74,"threads_started":1}
I20260806 18:34:45.683506 28193 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0a679fe952314ff8b936ca074c980c93" candidate_uuid: "63955300c20044878851d5e44e44155c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "885703862b27424890515f701c83aa68" is_pre_election: true
I20260806 18:34:45.683506 28334 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0a679fe952314ff8b936ca074c980c93" candidate_uuid: "63955300c20044878851d5e44e44155c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4575246688ab47bdb5abf0da5c702767" is_pre_election: true
I20260806 18:34:45.683656 28334 raft_consensus.cc:2454] T 0a679fe952314ff8b936ca074c980c93 P 4575246688ab47bdb5abf0da5c702767 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 63955300c20044878851d5e44e44155c in term 0.
I20260806 18:34:45.683656 28193 raft_consensus.cc:2454] T 0a679fe952314ff8b936ca074c980c93 P 885703862b27424890515f701c83aa68 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 63955300c20044878851d5e44e44155c in term 0.
I20260806 18:34:45.684039 28415 leader_election.cc:302] T 0a679fe952314ff8b936ca074c980c93 P 63955300c20044878851d5e44e44155c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4575246688ab47bdb5abf0da5c702767, 63955300c20044878851d5e44e44155c; no voters:
I20260806 18:34:45.684274 28565 raft_consensus.cc:2781] T 0a679fe952314ff8b936ca074c980c93 P 63955300c20044878851d5e44e44155c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 18:34:45.684338 28565 raft_consensus.cc:482] T 0a679fe952314ff8b936ca074c980c93 P 63955300c20044878851d5e44e44155c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 18:34:45.684377 28565 raft_consensus.cc:3037] T 0a679fe952314ff8b936ca074c980c93 P 63955300c20044878851d5e44e44155c [term 0 FOLLOWER]: Advancing to term 1
I20260806 18:34:45.685406 28565 raft_consensus.cc:504] T 0a679fe952314ff8b936ca074c980c93 P 63955300c20044878851d5e44e44155c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "885703862b27424890515f701c83aa68" member_type: VOTER last_known_addr { host: "127.19.194.1" port: 34859 } } peers { permanent_uuid: "4575246688ab47bdb5abf0da5c702767" member_type: VOTER last_known_addr { host: "127.19.194.2" port: 44765 } } peers { permanent_uuid: "63955300c20044878851d5e44e44155c" member_type: VOTER last_known_addr { host: "127.19.194.3" port: 46307 } }
I20260806 18:34:45.686028 28565 leader_election.cc:288] T 0a679fe952314ff8b936ca074c980c93 P 63955300c20044878851d5e44e44155c [CANDIDATE]: Term 1 election: Requested vote from peers 885703862b27424890515f701c83aa68 (127.19.194.1:34859), 4575246688ab47bdb5abf0da5c702767 (127.19.194.2:44765)
I20260806 18:34:45.686421 28334 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0a679fe952314ff8b936ca074c980c93" candidate_uuid: "63955300c20044878851d5e44e44155c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4575246688ab47bdb5abf0da5c702767"
I20260806 18:34:45.686421 28193 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0a679fe952314ff8b936ca074c980c93" candidate_uuid: "63955300c20044878851d5e44e44155c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "885703862b27424890515f701c83aa68"
I20260806 18:34:45.686558 28193 raft_consensus.cc:3037] T 0a679fe952314ff8b936ca074c980c93 P 885703862b27424890515f701c83aa68 [term 0 FOLLOWER]: Advancing to term 1
I20260806 18:34:45.686558 28334 raft_consensus.cc:3037] T 0a679fe952314ff8b936ca074c980c93 P 4575246688ab47bdb5abf0da5c702767 [term 0 FOLLOWER]: Advancing to term 1
I20260806 18:34:45.687914 28334 raft_consensus.cc:2454] T 0a679fe952314ff8b936ca074c980c93 P 4575246688ab47bdb5abf0da5c702767 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 63955300c20044878851d5e44e44155c in term 1.
I20260806 18:34:45.687978 28193 raft_consensus.cc:2454] T 0a679fe952314ff8b936ca074c980c93 P 885703862b27424890515f701c83aa68 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 63955300c20044878851d5e44e44155c in term 1.
I20260806 18:34:45.688282 28415 leader_election.cc:302] T 0a679fe952314ff8b936ca074c980c93 P 63955300c20044878851d5e44e44155c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 4575246688ab47bdb5abf0da5c702767, 63955300c20044878851d5e44e44155c, 885703862b27424890515f701c83aa68; no voters:
I20260806 18:34:45.688493 28565 raft_consensus.cc:2781] T 0a679fe952314ff8b936ca074c980c93 P 63955300c20044878851d5e44e44155c [term 1 FOLLOWER]: Leader election won for term 1
I20260806 18:34:45.688674 28565 raft_consensus.cc:686] T 0a679fe952314ff8b936ca074c980c93 P 63955300c20044878851d5e44e44155c [term 1 LEADER]: Becoming Leader. State: Replica: 63955300c20044878851d5e44e44155c, State: Running, Role: LEADER
I20260806 18:34:45.689237 28565 consensus_queue.cc:237] T 0a679fe952314ff8b936ca074c980c93 P 63955300c20044878851d5e44e44155c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "885703862b27424890515f701c83aa68" member_type: VOTER last_known_addr { host: "127.19.194.1" port: 34859 } } peers { permanent_uuid: "4575246688ab47bdb5abf0da5c702767" member_type: VOTER last_known_addr { host: "127.19.194.2" port: 44765 } } peers { permanent_uuid: "63955300c20044878851d5e44e44155c" member_type: VOTER last_known_addr { host: "127.19.194.3" port: 46307 } }
I20260806 18:34:45.691659 28035 catalog_manager.cc:5495] T 0a679fe952314ff8b936ca074c980c93 P 63955300c20044878851d5e44e44155c reported cstate change: term changed from 0 to 1, leader changed from <none> to 63955300c20044878851d5e44e44155c (127.19.194.3). New cstate: current_term: 1 leader_uuid: "63955300c20044878851d5e44e44155c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "885703862b27424890515f701c83aa68" member_type: VOTER last_known_addr { host: "127.19.194.1" port: 34859 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4575246688ab47bdb5abf0da5c702767" member_type: VOTER last_known_addr { host: "127.19.194.2" port: 44765 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "63955300c20044878851d5e44e44155c" member_type: VOTER last_known_addr { host: "127.19.194.3" port: 46307 } health_report { overall_health: HEALTHY } } }
I20260806 18:34:45.772845 28650 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:34:45.769044 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41998 (local address 127.19.194.3:46307)
0806 18:34:45.769381 (+ 337us) server_negotiation.cc:206] Beginning negotiation
0806 18:34:45.769396 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0806 18:34:45.769406 (+ 10us) server_negotiation.cc:407] Connection header received
0806 18:34:45.769474 (+ 68us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:34:45.769476 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:34:45.769534 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:34:45.769670 (+ 136us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 18:34:45.769948 (+ 278us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:34:45.770476 (+ 528us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:34:45.771317 (+ 841us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:34:45.771545 (+ 228us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:34:45.771660 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:34:45.771940 (+ 280us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 18:34:45.772428 (+ 488us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 18:34:45.772580 (+ 152us) server_negotiation.cc:1035] Waiting for connection context
0806 18:34:45.772635 (+ 55us) server_negotiation.cc:299] Negotiation successful
0806 18:34:45.772689 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":262,"thread_start_us":140,"threads_started":1}
I20260806 18:34:45.782356 28558 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:34:45.779225 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.194.3:36585 (local address 127.19.194.1:34859)
0806 18:34:45.779339 (+ 114us) server_negotiation.cc:206] Beginning negotiation
0806 18:34:45.779342 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 18:34:45.779356 (+ 14us) server_negotiation.cc:407] Connection header received
0806 18:34:45.779470 (+ 114us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:34:45.779473 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:34:45.779514 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:34:45.779574 (+ 60us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 18:34:45.779783 (+ 209us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:34:45.780275 (+ 492us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:34:45.781140 (+ 865us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:34:45.781426 (+ 286us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:34:45.781504 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:34:45.781797 (+ 293us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 18:34:45.781931 (+ 134us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 18:34:45.782050 (+ 119us) server_negotiation.cc:1035] Waiting for connection context
0806 18:34:45.782172 (+ 122us) server_negotiation.cc:299] Negotiation successful
0806 18:34:45.782215 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260806 18:34:45.782796 28653 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:34:45.779034 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.194.1:34859 (local address 127.19.194.3:36585)
0806 18:34:45.779304 (+ 270us) negotiation.cc:107] Waiting for socket to connect
0806 18:34:45.779317 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0806 18:34:45.779388 (+ 71us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:34:45.779601 (+ 213us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:34:45.779603 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:34:45.779608 (+ 5us) client_negotiation.cc:189] Negotiated authn=TOKEN
0806 18:34:45.779699 (+ 91us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:34:45.779703 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:34:45.780372 (+ 669us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:34:45.780374 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:34:45.781038 (+ 664us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:34:45.781047 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:34:45.781541 (+ 494us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:34:45.781544 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:34:45.781620 (+ 76us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:34:45.781661 (+ 41us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0806 18:34:45.782058 (+ 397us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0806 18:34:45.782067 (+ 9us) client_negotiation.cc:769] Sending connection context
0806 18:34:45.782149 (+ 82us) client_negotiation.cc:240] Negotiation successful
0806 18:34:45.782227 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":173,"thread_start_us":114,"threads_started":1}
I20260806 18:34:45.788230 28334 raft_consensus.cc:1261] T 0a679fe952314ff8b936ca074c980c93 P 4575246688ab47bdb5abf0da5c702767 [term 1 FOLLOWER]: Refusing update from remote peer 63955300c20044878851d5e44e44155c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 18:34:45.788473 28193 raft_consensus.cc:1261] T 0a679fe952314ff8b936ca074c980c93 P 885703862b27424890515f701c83aa68 [term 1 FOLLOWER]: Refusing update from remote peer 63955300c20044878851d5e44e44155c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 18:34:45.788733 28648 consensus_queue.cc:1035] T 0a679fe952314ff8b936ca074c980c93 P 63955300c20044878851d5e44e44155c [LEADER]: Connected to new peer: Peer: permanent_uuid: "4575246688ab47bdb5abf0da5c702767" member_type: VOTER last_known_addr { host: "127.19.194.2" port: 44765 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 18:34:45.788929 28565 consensus_queue.cc:1035] T 0a679fe952314ff8b936ca074c980c93 P 63955300c20044878851d5e44e44155c [LEADER]: Connected to new peer: Peer: permanent_uuid: "885703862b27424890515f701c83aa68" member_type: VOTER last_known_addr { host: "127.19.194.1" port: 34859 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 18:34:45.805001 28650 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:34:45.801643 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.194.1:51431 (local address 127.19.194.3:46307)
0806 18:34:45.801752 (+ 109us) server_negotiation.cc:206] Beginning negotiation
0806 18:34:45.801759 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 18:34:45.801964 (+ 205us) server_negotiation.cc:407] Connection header received
0806 18:34:45.802084 (+ 120us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:34:45.802087 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:34:45.802129 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:34:45.802216 (+ 87us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 18:34:45.802448 (+ 232us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:34:45.802937 (+ 489us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:34:45.803955 (+ 1018us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:34:45.804188 (+ 233us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:34:45.804302 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:34:45.804559 (+ 257us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 18:34:45.804652 (+ 93us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 18:34:45.804728 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0806 18:34:45.804825 (+ 97us) server_negotiation.cc:299] Negotiation successful
0806 18:34:45.804874 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260806 18:34:45.805025 28658 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:34:45.801581 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.194.3:46307 (local address 127.19.194.1:51431)
0806 18:34:45.801895 (+ 314us) negotiation.cc:107] Waiting for socket to connect
0806 18:34:45.801910 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0806 18:34:45.802002 (+ 92us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:34:45.802226 (+ 224us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:34:45.802229 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:34:45.802237 (+ 8us) client_negotiation.cc:189] Negotiated authn=TOKEN
0806 18:34:45.802338 (+ 101us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:34:45.802344 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:34:45.803080 (+ 736us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:34:45.803084 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:34:45.803819 (+ 735us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:34:45.803829 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:34:45.804305 (+ 476us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:34:45.804308 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:34:45.804395 (+ 87us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:34:45.804429 (+ 34us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0806 18:34:45.804748 (+ 319us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0806 18:34:45.804754 (+ 6us) client_negotiation.cc:769] Sending connection context
0806 18:34:45.804815 (+ 61us) client_negotiation.cc:240] Negotiation successful
0806 18:34:45.804896 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":206,"thread_start_us":72,"threads_started":1}
I20260806 18:34:45.827474 28033 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:46824:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 18:34:45.827663 28033 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:46824:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 18:34:45.829800 28033 catalog_manager.cc:5782] T 00000000000000000000000000000000 P d0282f2efb864823bbf90a21992ab39d: Sending DeleteTablet for 3 replicas of tablet 0a679fe952314ff8b936ca074c980c93
I20260806 18:34:45.830417 28173 tablet_service.cc:1514] Processing DeleteTablet for tablet 0a679fe952314ff8b936ca074c980c93 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 18:34:45 UTC) from {username='oryx', principal='oryx/127.19.194.62@KRBTEST.COM'} at 127.0.0.1:33838
I20260806 18:34:45.830446 28461 tablet_service.cc:1514] Processing DeleteTablet for tablet 0a679fe952314ff8b936ca074c980c93 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 18:34:45 UTC) from {username='oryx', principal='oryx/127.19.194.62@KRBTEST.COM'} at 127.0.0.1:41986
I20260806 18:34:45.830621 28585 tablet_replica.cc:330] T 0a679fe952314ff8b936ca074c980c93 P 885703862b27424890515f701c83aa68: stopping tablet replica
I20260806 18:34:45.830621 28583 tablet_replica.cc:330] T 0a679fe952314ff8b936ca074c980c93 P 63955300c20044878851d5e44e44155c: stopping tablet replica
I20260806 18:34:45.830726 28585 raft_consensus.cc:2229] T 0a679fe952314ff8b936ca074c980c93 P 885703862b27424890515f701c83aa68 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 18:34:45.830725 28583 raft_consensus.cc:2229] T 0a679fe952314ff8b936ca074c980c93 P 63955300c20044878851d5e44e44155c [term 1 LEADER]: Raft consensus shutting down.
I20260806 18:34:45.830828 28585 raft_consensus.cc:2258] T 0a679fe952314ff8b936ca074c980c93 P 885703862b27424890515f701c83aa68 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 18:34:45.831022 28583 raft_consensus.cc:2258] T 0a679fe952314ff8b936ca074c980c93 P 63955300c20044878851d5e44e44155c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 18:34:45.831190 20232 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJW69tn/build/debug/bin/kudu with pid 28108
I20260806 18:34:45.833258 28313 tablet_service.cc:1514] Processing DeleteTablet for tablet 0a679fe952314ff8b936ca074c980c93 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 18:34:45 UTC) from {username='oryx', principal='oryx/127.19.194.62@KRBTEST.COM'} at 127.0.0.1:51842
I20260806 18:34:45.833431 28584 tablet_replica.cc:330] T 0a679fe952314ff8b936ca074c980c93 P 4575246688ab47bdb5abf0da5c702767: stopping tablet replica
I20260806 18:34:45.833519 28584 raft_consensus.cc:2229] T 0a679fe952314ff8b936ca074c980c93 P 4575246688ab47bdb5abf0da5c702767 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 18:34:45.833618 28584 raft_consensus.cc:2258] T 0a679fe952314ff8b936ca074c980c93 P 4575246688ab47bdb5abf0da5c702767 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 18:34:45.834434 28583 ts_tablet_manager.cc:1905] T 0a679fe952314ff8b936ca074c980c93 P 63955300c20044878851d5e44e44155c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 18:34:45.835094 28584 ts_tablet_manager.cc:1905] T 0a679fe952314ff8b936ca074c980c93 P 4575246688ab47bdb5abf0da5c702767: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 18:34:45.837163 28583 ts_tablet_manager.cc:1918] T 0a679fe952314ff8b936ca074c980c93 P 63955300c20044878851d5e44e44155c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260806 18:34:45.837230 28583 log.cc:1196] T 0a679fe952314ff8b936ca074c980c93 P 63955300c20044878851d5e44e44155c: Deleting WAL directory at /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786041246094603-20232-0/minicluster-data/ts-2/wal/wals/0a679fe952314ff8b936ca074c980c93
I20260806 18:34:45.837495 28583 ts_tablet_manager.cc:1939] T 0a679fe952314ff8b936ca074c980c93 P 63955300c20044878851d5e44e44155c: Deleting consensus metadata
I20260806 18:34:45.837793 28584 ts_tablet_manager.cc:1918] T 0a679fe952314ff8b936ca074c980c93 P 4575246688ab47bdb5abf0da5c702767: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260806 18:34:45.837862 28584 log.cc:1196] T 0a679fe952314ff8b936ca074c980c93 P 4575246688ab47bdb5abf0da5c702767: Deleting WAL directory at /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786041246094603-20232-0/minicluster-data/ts-1/wal/wals/0a679fe952314ff8b936ca074c980c93
I20260806 18:34:45.838122 28584 ts_tablet_manager.cc:1939] T 0a679fe952314ff8b936ca074c980c93 P 4575246688ab47bdb5abf0da5c702767: Deleting consensus metadata
I20260806 18:34:45.838210 28019 catalog_manager.cc:4841] TS 63955300c20044878851d5e44e44155c (127.19.194.3:46307): tablet 0a679fe952314ff8b936ca074c980c93 (table test-table [id=856bb2f982d148748b19b894c6d127da]) successfully deleted
I20260806 18:34:45.838820 28021 catalog_manager.cc:4841] TS 4575246688ab47bdb5abf0da5c702767 (127.19.194.2:44765): tablet 0a679fe952314ff8b936ca074c980c93 (table test-table [id=856bb2f982d148748b19b894c6d127da]) successfully deleted
W20260806 18:34:45.843840 28022 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.19.194.1:34859 (error 108)
I20260806 18:34:45.844161 20232 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJW69tn/build/debug/bin/kudu with pid 28248
I20260806 18:34:45.847859 28662 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:34:45.847426 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.194.1:34859 (local address 127.0.0.1:33872)
0806 18:34:45.847698 (+ 272us) negotiation.cc:107] Waiting for socket to connect
0806 18:34:45.847777 (+ 79us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.194.1:34859: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":169,"thread_start_us":74,"threads_started":1}
W20260806 18:34:45.853376 28022 catalog_manager.cc:4584] TS 885703862b27424890515f701c83aa68 (127.19.194.1:34859): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 0a679fe952314ff8b936ca074c980c93: Network error: Client connection negotiation failed: client connection to 127.19.194.1:34859: connect: Connection refused (error 111)
I20260806 18:34:45.855285 20232 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJW69tn/build/debug/bin/kudu with pid 28397
I20260806 18:34:45.862210 20232 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJW69tn/build/debug/bin/kudu with pid 28003
2026-08-06T18:34:45Z chronyd exiting
[ OK ] SecurityITest.TestNonDefaultPrincipal (3228 ms)
[ RUN ] SecurityITest.TestNonExistentPrincipal
Loading random data
Initializing database '/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786041246094603-20232-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 18:34:45 dist-test-slave-1xrh krb5kdc[28665](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 18:34:45 dist-test-slave-1xrh krb5kdc[28665](info): set up 2 sockets
Aug 06 18:34:45 dist-test-slave-1xrh krb5kdc[28665](info): commencing operation
krb5kdc: starting...
W20260806 18:34:46.917062 20232 mini_kdc.cc:121] Time spent starting KDC: real 1.031s user 0.001s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786041246094603-20232-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786041246094603-20232-0/krb5kdc/test-user.keytab.
Aug 06 18:34:47 dist-test-slave-1xrh krb5kdc[28665](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041287, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T18:34:47Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T18:34:47Z Disabled control of system clock
WARNING: no policy specified for kudu/127.19.194.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.19.194.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.19.194.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786041246094603-20232-0/krb5kdc/kudu_127.19.194.62.keytab.
Entry for principal kudu/127.19.194.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786041246094603-20232-0/krb5kdc/kudu_127.19.194.62.keytab.
I20260806 18:34:47.040449 20232 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJW69tn/build/debug/bin/kudu
/tmp/dist-test-taskJW69tn/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786041246094603-20232-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786041246094603-20232-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786041246094603-20232-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786041246094603-20232-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.19.194.62:42793
--webserver_interface=127.19.194.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786041246094603-20232-0/krb5kdc/kudu_127.19.194.62.keytab
--principal=kudu/127.19.194.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.19.194.20:33419
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.19.194.62:42793
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--principal=oryx with env {KRB5CCNAME=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786041246094603-20232-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786041246094603-20232-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786041246094603-20232-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 18:34:47.146610 28678 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 18:34:47.146847 28678 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 18:34:47.146885 28678 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 18:34:47.151417 28678 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 18:34:47.151466 28678 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 18:34:47.151486 28678 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 18:34:47.151502 28678 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 18:34:47.151546 28678 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 18:34:47.156663 28678 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.194.20:33419
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786041246094603-20232-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786041246094603-20232-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.19.194.62:42793
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786041246094603-20232-0/krb5kdc/kudu_127.19.194.62.keytab
--principal=oryx
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.19.194.62:42793
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786041246094603-20232-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.19.194.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28678
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786041246094603-20232-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 17:59:13 UTC on ccycloud.quasar-wvsnzx.root.comops.site
build id 4571028
I20260806 18:34:47.157644 28678 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 18:34:47.158645 28678 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 18:34:47.166827 28684 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:34:47.166929 28687 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:34:47.166836 28685 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 18:34:47.167220 28678 server_base.cc:1034] running on GCE node
I20260806 18:34:47.167745 28678 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 18:34:47.168800 28678 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 18:34:47.169998 28678 hybrid_clock.cc:648] HybridClock initialized: now 1786041287169983 us; error 31 us; skew 500 ppm
Runtime error: kinit failed: unable to login from keytab: Keytab contains no suitable keys for oryx@KRBTEST.COM
2026-08-06T18:34:47Z chronyd exiting
[ OK ] SecurityITest.TestNonExistentPrincipal (1313 ms)
[ RUN ] SecurityITest.TestMismatchingPrincipals
Loading random data
Initializing database '/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786041246094603-20232-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 18:34:47 dist-test-slave-1xrh krb5kdc[28692](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 18:34:47 dist-test-slave-1xrh krb5kdc[28692](info): set up 2 sockets
Aug 06 18:34:47 dist-test-slave-1xrh krb5kdc[28692](info): commencing operation
krb5kdc: starting...
W20260806 18:34:48.247694 20232 mini_kdc.cc:121] Time spent starting KDC: real 1.048s user 0.004s sys 0.002s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786041246094603-20232-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786041246094603-20232-0/krb5kdc/test-user.keytab.
Aug 06 18:34:48 dist-test-slave-1xrh krb5kdc[28692](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041288, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T18:34:48Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T18:34:48Z Disabled control of system clock
WARNING: no policy specified for kudu/127.19.194.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.19.194.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.19.194.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786041246094603-20232-0/krb5kdc/kudu_127.19.194.62.keytab.
Entry for principal kudu/127.19.194.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786041246094603-20232-0/krb5kdc/kudu_127.19.194.62.keytab.
I20260806 18:34:48.370198 20232 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJW69tn/build/debug/bin/kudu
/tmp/dist-test-taskJW69tn/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786041246094603-20232-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786041246094603-20232-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786041246094603-20232-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786041246094603-20232-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.19.194.62:44589
--webserver_interface=127.19.194.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786041246094603-20232-0/krb5kdc/kudu_127.19.194.62.keytab
--principal=kudu/127.19.194.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.19.194.20:42015
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.19.194.62:44589
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786041246094603-20232-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786041246094603-20232-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786041246094603-20232-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 18:34:48.478457 28705 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 18:34:48.478685 28705 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 18:34:48.478724 28705 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 18:34:48.482529 28705 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 18:34:48.482576 28705 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 18:34:48.482594 28705 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 18:34:48.482609 28705 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 18:34:48.482621 28705 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 18:34:48.487671 28705 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.194.20:42015
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786041246094603-20232-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786041246094603-20232-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.19.194.62:44589
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786041246094603-20232-0/krb5kdc/kudu_127.19.194.62.keytab
--principal=kudu/127.19.194.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.19.194.62:44589
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786041246094603-20232-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.19.194.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28705
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786041246094603-20232-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 17:59:13 UTC on ccycloud.quasar-wvsnzx.root.comops.site
build id 4571028
I20260806 18:34:48.488610 28705 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 18:34:48.489516 28705 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260806 18:34:48.497800 28705 server_base.cc:1034] running on GCE node
W20260806 18:34:48.497741 28710 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:34:48.497762 28714 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:34:48.497761 28712 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 18:34:48.498492 28705 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 18:34:48.499606 28705 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 18:34:48.500787 28705 hybrid_clock.cc:648] HybridClock initialized: now 1786041288500757 us; error 43 us; skew 500 ppm
Aug 06 18:34:48 dist-test-slave-1xrh krb5kdc[28692](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041288, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.194.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 18:34:48.503556 28705 init.cc:377] Logged in from keytab as kudu/127.19.194.62@KRBTEST.COM (short username kudu)
I20260806 18:34:48.504542 28705 webserver.cc:460] Webserver started at http://127.19.194.62:40121/ using document root <none> and password file <none>
I20260806 18:34:48.505100 28705 fs_manager.cc:359] Metadata directory not provided
I20260806 18:34:48.505193 28705 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 18:34:48.505419 28705 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 18:34:48.507447 28705 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786041246094603-20232-0/minicluster-data/master-0/data/instance:
uuid: "189709350f1345d8a5ecbcfd6d1c55f2"
format_stamp: "Formatted at 2026-08-06 18:34:48 on dist-test-slave-1xrh"
server_key: "ec57af6bdc7ef463964fbaca1ee22045"
server_key_iv: "870fe58f6097e3b9c83fbb94f87dd50c"
server_key_version: "encryptionkey@0"
I20260806 18:34:48.508023 28705 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786041246094603-20232-0/minicluster-data/master-0/wal/instance:
uuid: "189709350f1345d8a5ecbcfd6d1c55f2"
format_stamp: "Formatted at 2026-08-06 18:34:48 on dist-test-slave-1xrh"
server_key: "ec57af6bdc7ef463964fbaca1ee22045"
server_key_iv: "870fe58f6097e3b9c83fbb94f87dd50c"
server_key_version: "encryptionkey@0"
I20260806 18:34:48.511502 28705 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.002s sys 0.001s
I20260806 18:34:48.513561 28721 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 18:34:48.514576 28705 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260806 18:34:48.514720 28705 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786041246094603-20232-0/minicluster-data/master-0/data,/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786041246094603-20232-0/minicluster-data/master-0/wal
uuid: "189709350f1345d8a5ecbcfd6d1c55f2"
format_stamp: "Formatted at 2026-08-06 18:34:48 on dist-test-slave-1xrh"
server_key: "ec57af6bdc7ef463964fbaca1ee22045"
server_key_iv: "870fe58f6097e3b9c83fbb94f87dd50c"
server_key_version: "encryptionkey@0"
I20260806 18:34:48.514824 28705 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786041246094603-20232-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786041246094603-20232-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786041246094603-20232-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 18:34:48.542232 28705 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 18:34:48.545095 28705 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 18:34:48.545297 28705 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 18:34:48.552712 28705 rpc_server.cc:307] RPC server started. Bound to: 127.19.194.62:44589
I20260806 18:34:48.552738 28773 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.194.62:44589 every 8 connection(s)
I20260806 18:34:48.553633 28705 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786041246094603-20232-0/minicluster-data/master-0/data/info.pb
I20260806 18:34:48.555704 20232 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJW69tn/build/debug/bin/kudu as pid 28705
I20260806 18:34:48.555826 20232 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786041246094603-20232-0/minicluster-data/master-0/wal/instance
I20260806 18:34:48.556182 20232 external_mini_cluster.cc:1428] Setting key c67d8541f654de49bc6590e034c80a6f
I20260806 18:34:48.556658 28774 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 18:34:48.563397 28774 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 189709350f1345d8a5ecbcfd6d1c55f2: Bootstrap starting.
Aug 06 18:34:48 dist-test-slave-1xrh krb5kdc[28692](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041288, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.19.194.62@KRBTEST.COM
I20260806 18:34:48.566134 28774 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 189709350f1345d8a5ecbcfd6d1c55f2: Neither blocks nor log segments found. Creating new log.
I20260806 18:34:48.566845 28774 log.cc:824] T 00000000000000000000000000000000 P 189709350f1345d8a5ecbcfd6d1c55f2: Log is configured to *not* fsync() on all Append() calls
I20260806 18:34:48.569021 28774 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 189709350f1345d8a5ecbcfd6d1c55f2: No bootstrap required, opened a new log
I20260806 18:34:48.571525 28777 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:34:48.557834 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38098 (local address 127.19.194.62:44589)
0806 18:34:48.558236 (+ 402us) server_negotiation.cc:206] Beginning negotiation
0806 18:34:48.558251 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0806 18:34:48.558285 (+ 34us) server_negotiation.cc:407] Connection header received
0806 18:34:48.559267 (+ 982us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:34:48.559285 (+ 18us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:34:48.559588 (+ 303us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:34:48.559843 (+ 255us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:34:48.560792 (+ 949us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:34:48.561734 (+ 942us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:34:48.562920 (+ 1186us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:34:48.563378 (+ 458us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:34:48.563542 (+ 164us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:34:48.566267 (+ 2725us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:34:48.566275 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:34:48.566295 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 18:34:48.566332 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:34:48.568340 (+ 2008us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:34:48.568776 (+ 436us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:34:48.568781 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:34:48.568787 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:34:48.568936 (+ 149us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:34:48.569228 (+ 292us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:34:48.569231 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:34:48.569233 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:34:48.569731 (+ 498us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:34:48.569923 (+ 192us) server_negotiation.cc:1035] Waiting for connection context
0806 18:34:48.570179 (+ 256us) server_negotiation.cc:299] Negotiation successful
0806 18:34:48.570456 (+ 277us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":264,"thread_start_us":149,"threads_started":1}
I20260806 18:34:48.573391 28774 raft_consensus.cc:348] T 00000000000000000000000000000000 P 189709350f1345d8a5ecbcfd6d1c55f2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "189709350f1345d8a5ecbcfd6d1c55f2" member_type: VOTER last_known_addr { host: "127.19.194.62" port: 44589 } }
I20260806 18:34:48.573745 28774 raft_consensus.cc:374] T 00000000000000000000000000000000 P 189709350f1345d8a5ecbcfd6d1c55f2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 18:34:48.573810 28774 raft_consensus.cc:729] T 00000000000000000000000000000000 P 189709350f1345d8a5ecbcfd6d1c55f2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 189709350f1345d8a5ecbcfd6d1c55f2, State: Initialized, Role: FOLLOWER
I20260806 18:34:48.574204 28774 consensus_queue.cc:260] T 00000000000000000000000000000000 P 189709350f1345d8a5ecbcfd6d1c55f2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "189709350f1345d8a5ecbcfd6d1c55f2" member_type: VOTER last_known_addr { host: "127.19.194.62" port: 44589 } }
I20260806 18:34:48.574344 28774 raft_consensus.cc:388] T 00000000000000000000000000000000 P 189709350f1345d8a5ecbcfd6d1c55f2 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 18:34:48.574409 28774 raft_consensus.cc:482] T 00000000000000000000000000000000 P 189709350f1345d8a5ecbcfd6d1c55f2 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 18:34:48.574506 28774 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 189709350f1345d8a5ecbcfd6d1c55f2 [term 0 FOLLOWER]: Advancing to term 1
I20260806 18:34:48.575443 28774 raft_consensus.cc:504] T 00000000000000000000000000000000 P 189709350f1345d8a5ecbcfd6d1c55f2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "189709350f1345d8a5ecbcfd6d1c55f2" member_type: VOTER last_known_addr { host: "127.19.194.62" port: 44589 } }
I20260806 18:34:48.575767 28774 leader_election.cc:302] T 00000000000000000000000000000000 P 189709350f1345d8a5ecbcfd6d1c55f2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 189709350f1345d8a5ecbcfd6d1c55f2; no voters:
I20260806 18:34:48.576058 28774 leader_election.cc:288] T 00000000000000000000000000000000 P 189709350f1345d8a5ecbcfd6d1c55f2 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 18:34:48.576193 28779 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 189709350f1345d8a5ecbcfd6d1c55f2 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 18:34:48.576517 28779 raft_consensus.cc:686] T 00000000000000000000000000000000 P 189709350f1345d8a5ecbcfd6d1c55f2 [term 1 LEADER]: Becoming Leader. State: Replica: 189709350f1345d8a5ecbcfd6d1c55f2, State: Running, Role: LEADER
I20260806 18:34:48.576843 28779 consensus_queue.cc:237] T 00000000000000000000000000000000 P 189709350f1345d8a5ecbcfd6d1c55f2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "189709350f1345d8a5ecbcfd6d1c55f2" member_type: VOTER last_known_addr { host: "127.19.194.62" port: 44589 } }
I20260806 18:34:48.577291 28774 sys_catalog.cc:564] T 00000000000000000000000000000000 P 189709350f1345d8a5ecbcfd6d1c55f2 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 18:34:48.577967 28779 sys_catalog.cc:455] T 00000000000000000000000000000000 P 189709350f1345d8a5ecbcfd6d1c55f2 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 189709350f1345d8a5ecbcfd6d1c55f2. Latest consensus state: current_term: 1 leader_uuid: "189709350f1345d8a5ecbcfd6d1c55f2" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "189709350f1345d8a5ecbcfd6d1c55f2" member_type: VOTER last_known_addr { host: "127.19.194.62" port: 44589 } } }
I20260806 18:34:48.578168 28779 sys_catalog.cc:458] T 00000000000000000000000000000000 P 189709350f1345d8a5ecbcfd6d1c55f2 [sys.catalog]: This master's current role is: LEADER
I20260806 18:34:48.578471 28780 sys_catalog.cc:455] T 00000000000000000000000000000000 P 189709350f1345d8a5ecbcfd6d1c55f2 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "189709350f1345d8a5ecbcfd6d1c55f2" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "189709350f1345d8a5ecbcfd6d1c55f2" member_type: VOTER last_known_addr { host: "127.19.194.62" port: 44589 } } }
I20260806 18:34:48.578675 28780 sys_catalog.cc:458] T 00000000000000000000000000000000 P 189709350f1345d8a5ecbcfd6d1c55f2 [sys.catalog]: This master's current role is: LEADER
I20260806 18:34:48.578611 28788 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 18:34:48.581516 28788 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 18:34:48.586229 28788 catalog_manager.cc:1347] Generated new cluster ID: ad68f8ee2d84469caadc916bd02be3df
I20260806 18:34:48.586290 28788 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 18:34:48.600646 28788 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 18:34:48.601688 28788 catalog_manager.cc:1504] Loading token signing keys...
I20260806 18:34:48.610044 28788 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 189709350f1345d8a5ecbcfd6d1c55f2: Generated new TSK 0
I20260806 18:34:48.610627 28788 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.19.194.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.19.194.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.19.194.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786041246094603-20232-0/krb5kdc/kudu_127.19.194.1.keytab.
Entry for principal kudu/127.19.194.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786041246094603-20232-0/krb5kdc/kudu_127.19.194.1.keytab.
I20260806 18:34:48.638486 20232 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJW69tn/build/debug/bin/kudu
/tmp/dist-test-taskJW69tn/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786041246094603-20232-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786041246094603-20232-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786041246094603-20232-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786041246094603-20232-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.19.194.1:0
--local_ip_for_outbound_sockets=127.19.194.1
--webserver_interface=127.19.194.1
--webserver_port=0
--tserver_master_addrs=127.19.194.62:44589
--keytab_file=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786041246094603-20232-0/krb5kdc/kudu_127.19.194.1.keytab
--principal=kudu/127.19.194.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.19.194.20:42015
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786041246094603-20232-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786041246094603-20232-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786041246094603-20232-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 18:34:48.747074 28800 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 18:34:48.747320 28800 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 18:34:48.747393 28800 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 18:34:48.751405 28800 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 18:34:48.751454 28800 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 18:34:48.751557 28800 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.19.194.1
I20260806 18:34:48.756697 28800 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.194.20:42015
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786041246094603-20232-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786041246094603-20232-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786041246094603-20232-0/krb5kdc/kudu_127.19.194.1.keytab
--principal=kudu/127.19.194.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.19.194.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786041246094603-20232-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.19.194.1
--webserver_port=0
--tserver_master_addrs=127.19.194.62:44589
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28800
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.19.194.1
--log_dir=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786041246094603-20232-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 17:59:13 UTC on ccycloud.quasar-wvsnzx.root.comops.site
build id 4571028
I20260806 18:34:48.757668 28800 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 18:34:48.758623 28800 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 18:34:48.767807 28806 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:34:48.767807 28808 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:34:48.767874 28805 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 18:34:48.767935 28800 server_base.cc:1034] running on GCE node
I20260806 18:34:48.768466 28800 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 18:34:48.769143 28800 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 18:34:48.770327 28800 hybrid_clock.cc:648] HybridClock initialized: now 1786041288770297 us; error 42 us; skew 500 ppm
Aug 06 18:34:48 dist-test-slave-1xrh krb5kdc[28692](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041288, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.194.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 18:34:48.773128 28800 init.cc:377] Logged in from keytab as kudu/127.19.194.1@KRBTEST.COM (short username kudu)
I20260806 18:34:48.774009 28800 webserver.cc:460] Webserver started at http://127.19.194.1:40065/ using document root <none> and password file <none>
I20260806 18:34:48.774487 28800 fs_manager.cc:359] Metadata directory not provided
I20260806 18:34:48.774544 28800 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 18:34:48.774701 28800 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 18:34:48.776679 28800 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786041246094603-20232-0/minicluster-data/ts-0/data/instance:
uuid: "1c30e79bd0454f94b1b6fe14f5c23888"
format_stamp: "Formatted at 2026-08-06 18:34:48 on dist-test-slave-1xrh"
server_key: "b2a1925ee37bfff885c3d4e0ce6936de"
server_key_iv: "68870f74d74d6581a080639467947330"
server_key_version: "encryptionkey@0"
I20260806 18:34:48.777243 28800 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786041246094603-20232-0/minicluster-data/ts-0/wal/instance:
uuid: "1c30e79bd0454f94b1b6fe14f5c23888"
format_stamp: "Formatted at 2026-08-06 18:34:48 on dist-test-slave-1xrh"
server_key: "b2a1925ee37bfff885c3d4e0ce6936de"
server_key_iv: "68870f74d74d6581a080639467947330"
server_key_version: "encryptionkey@0"
I20260806 18:34:48.780871 28800 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.005s sys 0.000s
I20260806 18:34:48.783202 28815 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 18:34:48.784377 28800 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260806 18:34:48.784473 28800 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786041246094603-20232-0/minicluster-data/ts-0/data,/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786041246094603-20232-0/minicluster-data/ts-0/wal
uuid: "1c30e79bd0454f94b1b6fe14f5c23888"
format_stamp: "Formatted at 2026-08-06 18:34:48 on dist-test-slave-1xrh"
server_key: "b2a1925ee37bfff885c3d4e0ce6936de"
server_key_iv: "68870f74d74d6581a080639467947330"
server_key_version: "encryptionkey@0"
I20260806 18:34:48.784552 28800 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786041246094603-20232-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786041246094603-20232-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786041246094603-20232-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 18:34:48.808751 28800 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 18:34:48.811959 28800 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 18:34:48.812117 28800 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 18:34:48.812654 28800 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 18:34:48.813372 28800 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 18:34:48.813436 28800 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 18:34:48.813474 28800 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 18:34:48.813495 28800 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 18:34:48.822741 28800 rpc_server.cc:307] RPC server started. Bound to: 127.19.194.1:44719
I20260806 18:34:48.822773 28929 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.194.1:44719 every 8 connection(s)
I20260806 18:34:48.823648 28800 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786041246094603-20232-0/minicluster-data/ts-0/data/info.pb
I20260806 18:34:48.824446 20232 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJW69tn/build/debug/bin/kudu as pid 28800
I20260806 18:34:48.824586 20232 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786041246094603-20232-0/minicluster-data/ts-0/wal/instance
I20260806 18:34:48.824873 20232 external_mini_cluster.cc:1428] Setting key 988bb874c951d5d2afe9fecae4431cf4
Aug 06 18:34:48 dist-test-slave-1xrh krb5kdc[28692](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041288, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.194.1@KRBTEST.COM for kudu/127.19.194.62@KRBTEST.COM
WARNING: no policy specified for kudu/127.19.194.2@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.19.194.2@KRBTEST.COM" created.
I20260806 18:34:48.838773 28777 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:34:48.825527 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.194.1:54055 (local address 127.19.194.62:44589)
0806 18:34:48.825661 (+ 134us) server_negotiation.cc:206] Beginning negotiation
0806 18:34:48.825668 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 18:34:48.826701 (+ 1033us) server_negotiation.cc:407] Connection header received
0806 18:34:48.828685 (+ 1984us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:34:48.828691 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:34:48.828756 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:34:48.828856 (+ 100us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:34:48.829999 (+ 1143us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:34:48.830901 (+ 902us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:34:48.832166 (+ 1265us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:34:48.832627 (+ 461us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:34:48.832787 (+ 160us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:34:48.835236 (+ 2449us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:34:48.835243 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:34:48.835249 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 18:34:48.835285 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:34:48.836646 (+ 1361us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:34:48.837127 (+ 481us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:34:48.837129 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:34:48.837131 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:34:48.837205 (+ 74us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:34:48.837506 (+ 301us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:34:48.837508 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:34:48.837509 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:34:48.837703 (+ 194us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:34:48.837776 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0806 18:34:48.838366 (+ 590us) server_negotiation.cc:299] Negotiation successful
0806 18:34:48.838514 (+ 148us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260806 18:34:48.839694 28932 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:34:48.825955 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.194.62:44589 (local address 127.19.194.1:54055)
0806 18:34:48.826522 (+ 567us) negotiation.cc:107] Waiting for socket to connect
0806 18:34:48.826579 (+ 57us) client_negotiation.cc:174] Beginning negotiation
0806 18:34:48.828381 (+ 1802us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:34:48.829016 (+ 635us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:34:48.829022 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:34:48.829360 (+ 338us) client_negotiation.cc:189] Negotiated authn=SASL
0806 18:34:48.829824 (+ 464us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:34:48.829837 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:34:48.831040 (+ 1203us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:34:48.831042 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:34:48.832030 (+ 988us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:34:48.832040 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:34:48.832808 (+ 768us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:34:48.832811 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:34:48.832945 (+ 134us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:34:48.833592 (+ 647us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 18:34:48.833609 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0806 18:34:48.835043 (+ 1434us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 18:34:48.836757 (+ 1714us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:34:48.836762 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:34:48.836773 (+ 11us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:34:48.837027 (+ 254us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:34:48.837283 (+ 256us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:34:48.837286 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:34:48.837287 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:34:48.837415 (+ 128us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:34:48.837798 (+ 383us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 18:34:48.837804 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 18:34:48.838035 (+ 231us) client_negotiation.cc:769] Sending connection context
0806 18:34:48.838277 (+ 242us) client_negotiation.cc:240] Negotiation successful
0806 18:34:48.838591 (+ 314us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":314,"thread_start_us":147,"threads_started":1}
I20260806 18:34:48.841034 28930 heartbeater.cc:347] Connected to a master server at 127.19.194.62:44589
I20260806 18:34:48.841310 28930 heartbeater.cc:464] Registering TS with master...
I20260806 18:34:48.841809 28930 heartbeater.cc:511] Master 127.19.194.62:44589 requested a full tablet report, sending...
I20260806 18:34:48.843600 28738 ts_manager.cc:194] Registered new tserver with Master: 1c30e79bd0454f94b1b6fe14f5c23888 (127.19.194.1:44719)
I20260806 18:34:48.845403 28738 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.19.194.1@KRBTEST.COM'} at 127.19.194.1:54055
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.19.194.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786041246094603-20232-0/krb5kdc/kudu_127.19.194.2.keytab.
Entry for principal kudu/127.19.194.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786041246094603-20232-0/krb5kdc/kudu_127.19.194.2.keytab.
I20260806 18:34:48.855934 20232 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJW69tn/build/debug/bin/kudu
/tmp/dist-test-taskJW69tn/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786041246094603-20232-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786041246094603-20232-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786041246094603-20232-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786041246094603-20232-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.19.194.2:0
--local_ip_for_outbound_sockets=127.19.194.2
--webserver_interface=127.19.194.2
--webserver_port=0
--tserver_master_addrs=127.19.194.62:44589
--keytab_file=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786041246094603-20232-0/krb5kdc/kudu_127.19.194.2.keytab
--principal=kudu/127.19.194.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.19.194.20:42015
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786041246094603-20232-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786041246094603-20232-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786041246094603-20232-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 18:34:48.964887 28935 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 18:34:48.965152 28935 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 18:34:48.965216 28935 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 18:34:48.969125 28935 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 18:34:48.969175 28935 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 18:34:48.969230 28935 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.19.194.2
I20260806 18:34:48.974241 28935 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.194.20:42015
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786041246094603-20232-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786041246094603-20232-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786041246094603-20232-0/krb5kdc/kudu_127.19.194.2.keytab
--principal=kudu/127.19.194.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.19.194.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786041246094603-20232-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.19.194.2
--webserver_port=0
--tserver_master_addrs=127.19.194.62:44589
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28935
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.19.194.2
--log_dir=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786041246094603-20232-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 17:59:13 UTC on ccycloud.quasar-wvsnzx.root.comops.site
build id 4571028
I20260806 18:34:48.975241 28935 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 18:34:48.976214 28935 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 18:34:48.985988 28943 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:34:48.985984 28941 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:34:48.985984 28940 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 18:34:48.986218 28935 server_base.cc:1034] running on GCE node
I20260806 18:34:48.986704 28935 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 18:34:48.987389 28935 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 18:34:48.988550 28935 hybrid_clock.cc:648] HybridClock initialized: now 1786041288988537 us; error 24 us; skew 500 ppm
Aug 06 18:34:48 dist-test-slave-1xrh krb5kdc[28692](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041288, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.194.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 18:34:48.998070 28935 init.cc:377] Logged in from keytab as kudu/127.19.194.2@KRBTEST.COM (short username kudu)
I20260806 18:34:48.999039 28935 webserver.cc:460] Webserver started at http://127.19.194.2:34001/ using document root <none> and password file <none>
I20260806 18:34:48.999627 28935 fs_manager.cc:359] Metadata directory not provided
I20260806 18:34:48.999725 28935 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 18:34:48.999963 28935 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 18:34:49.002148 28935 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786041246094603-20232-0/minicluster-data/ts-1/data/instance:
uuid: "e6f6733b4f044979bb83e56cb63cbcb6"
format_stamp: "Formatted at 2026-08-06 18:34:48 on dist-test-slave-1xrh"
server_key: "ef681e181beb8b9b22e5068ee96f7111"
server_key_iv: "76d6230dca250e32308b2dba34761b9d"
server_key_version: "encryptionkey@0"
I20260806 18:34:49.002740 28935 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786041246094603-20232-0/minicluster-data/ts-1/wal/instance:
uuid: "e6f6733b4f044979bb83e56cb63cbcb6"
format_stamp: "Formatted at 2026-08-06 18:34:48 on dist-test-slave-1xrh"
server_key: "ef681e181beb8b9b22e5068ee96f7111"
server_key_iv: "76d6230dca250e32308b2dba34761b9d"
server_key_version: "encryptionkey@0"
I20260806 18:34:49.006596 28935 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260806 18:34:49.009025 28950 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 18:34:49.010255 28935 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260806 18:34:49.010401 28935 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786041246094603-20232-0/minicluster-data/ts-1/data,/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786041246094603-20232-0/minicluster-data/ts-1/wal
uuid: "e6f6733b4f044979bb83e56cb63cbcb6"
format_stamp: "Formatted at 2026-08-06 18:34:48 on dist-test-slave-1xrh"
server_key: "ef681e181beb8b9b22e5068ee96f7111"
server_key_iv: "76d6230dca250e32308b2dba34761b9d"
server_key_version: "encryptionkey@0"
I20260806 18:34:49.010512 28935 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786041246094603-20232-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786041246094603-20232-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786041246094603-20232-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 18:34:49.032732 28935 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 18:34:49.035564 28935 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 18:34:49.035774 28935 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 18:34:49.036365 28935 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 18:34:49.037186 28935 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 18:34:49.037261 28935 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 18:34:49.037328 28935 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 18:34:49.037379 28935 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 18:34:49.046334 28935 rpc_server.cc:307] RPC server started. Bound to: 127.19.194.2:33567
I20260806 18:34:49.046361 29066 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.194.2:33567 every 8 connection(s)
I20260806 18:34:49.047226 28935 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786041246094603-20232-0/minicluster-data/ts-1/data/info.pb
I20260806 18:34:49.052301 20232 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJW69tn/build/debug/bin/kudu as pid 28935
I20260806 18:34:49.052420 20232 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786041246094603-20232-0/minicluster-data/ts-1/wal/instance
I20260806 18:34:49.052806 20232 external_mini_cluster.cc:1428] Setting key c542343231c1a1b108cf2ca4c3455b3b
Aug 06 18:34:49 dist-test-slave-1xrh krb5kdc[28692](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041288, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.194.2@KRBTEST.COM for kudu/127.19.194.62@KRBTEST.COM
I20260806 18:34:49.061842 28777 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:34:49.049029 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.194.2:59977 (local address 127.19.194.62:44589)
0806 18:34:49.049163 (+ 134us) server_negotiation.cc:206] Beginning negotiation
0806 18:34:49.049170 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 18:34:49.049956 (+ 786us) server_negotiation.cc:407] Connection header received
0806 18:34:49.051294 (+ 1338us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:34:49.051298 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:34:49.051356 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:34:49.051449 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:34:49.052785 (+ 1336us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:34:49.053366 (+ 581us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:34:49.054725 (+ 1359us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:34:49.054934 (+ 209us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:34:49.055017 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:34:49.058101 (+ 3084us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:34:49.058108 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:34:49.058113 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 18:34:49.058144 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:34:49.059843 (+ 1699us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:34:49.060320 (+ 477us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:34:49.060325 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:34:49.060327 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:34:49.060437 (+ 110us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:34:49.060752 (+ 315us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:34:49.060758 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:34:49.060761 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:34:49.061014 (+ 253us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:34:49.061124 (+ 110us) server_negotiation.cc:1035] Waiting for connection context
0806 18:34:49.061537 (+ 413us) server_negotiation.cc:299] Negotiation successful
0806 18:34:49.061674 (+ 137us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260806 18:34:49.062471 29069 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:34:49.049360 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.194.62:44589 (local address 127.19.194.2:59977)
0806 18:34:49.049797 (+ 437us) negotiation.cc:107] Waiting for socket to connect
0806 18:34:49.049837 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0806 18:34:49.050970 (+ 1133us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:34:49.051608 (+ 638us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:34:49.051614 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:34:49.051927 (+ 313us) client_negotiation.cc:189] Negotiated authn=SASL
0806 18:34:49.052611 (+ 684us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:34:49.052634 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:34:49.053475 (+ 841us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:34:49.053478 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:34:49.054593 (+ 1115us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:34:49.054607 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:34:49.055097 (+ 490us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:34:49.055104 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:34:49.055363 (+ 259us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:34:49.055914 (+ 551us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 18:34:49.055939 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0806 18:34:49.057937 (+ 1998us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 18:34:49.059952 (+ 2015us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:34:49.059956 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:34:49.059967 (+ 11us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:34:49.060191 (+ 224us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:34:49.060535 (+ 344us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:34:49.060537 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:34:49.060539 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:34:49.060653 (+ 114us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:34:49.061115 (+ 462us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 18:34:49.061119 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 18:34:49.061307 (+ 188us) client_negotiation.cc:769] Sending connection context
0806 18:34:49.061441 (+ 134us) client_negotiation.cc:240] Negotiation successful
0806 18:34:49.061664 (+ 223us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":266,"thread_start_us":100,"threads_started":1}
WARNING: no policy specified for kudu/127.19.194.3@KRBTEST.COM; defaulting to no policy
I20260806 18:34:49.063591 29067 heartbeater.cc:347] Connected to a master server at 127.19.194.62:44589
I20260806 18:34:49.063805 29067 heartbeater.cc:464] Registering TS with master...
I20260806 18:34:49.064319 29067 heartbeater.cc:511] Master 127.19.194.62:44589 requested a full tablet report, sending...
I20260806 18:34:49.065352 28738 ts_manager.cc:194] Registered new tserver with Master: e6f6733b4f044979bb83e56cb63cbcb6 (127.19.194.2:33567)
I20260806 18:34:49.065972 28738 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.19.194.2@KRBTEST.COM'} at 127.19.194.2:59977
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.19.194.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.19.194.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786041246094603-20232-0/krb5kdc/kudu_127.19.194.3.keytab.
Entry for principal kudu/127.19.194.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786041246094603-20232-0/krb5kdc/kudu_127.19.194.3.keytab.
I20260806 18:34:49.082844 20232 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJW69tn/build/debug/bin/kudu
/tmp/dist-test-taskJW69tn/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786041246094603-20232-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786041246094603-20232-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786041246094603-20232-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786041246094603-20232-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.19.194.3:0
--local_ip_for_outbound_sockets=127.19.194.3
--webserver_interface=127.19.194.3
--webserver_port=0
--tserver_master_addrs=127.19.194.62:44589
--keytab_file=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786041246094603-20232-0/krb5kdc/kudu_127.19.194.3.keytab
--principal=kudu/127.19.194.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.19.194.20:42015
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786041246094603-20232-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786041246094603-20232-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786041246094603-20232-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 18:34:49.191032 29072 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 18:34:49.191284 29072 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 18:34:49.191327 29072 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 18:34:49.195174 29072 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 18:34:49.195225 29072 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 18:34:49.195353 29072 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.19.194.3
I20260806 18:34:49.200562 29072 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.194.20:42015
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786041246094603-20232-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786041246094603-20232-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786041246094603-20232-0/krb5kdc/kudu_127.19.194.3.keytab
--principal=kudu/127.19.194.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.19.194.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786041246094603-20232-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.19.194.3
--webserver_port=0
--tserver_master_addrs=127.19.194.62:44589
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29072
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.19.194.3
--log_dir=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786041246094603-20232-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 17:59:13 UTC on ccycloud.quasar-wvsnzx.root.comops.site
build id 4571028
I20260806 18:34:49.201545 29072 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 18:34:49.202518 29072 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260806 18:34:49.212455 29072 server_base.cc:1034] running on GCE node
W20260806 18:34:49.212468 29080 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:34:49.212459 29078 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:34:49.212471 29077 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 18:34:49.213091 29072 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 18:34:49.213732 29072 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 18:34:49.214941 29072 hybrid_clock.cc:648] HybridClock initialized: now 1786041289214909 us; error 39 us; skew 500 ppm
Aug 06 18:34:49 dist-test-slave-1xrh krb5kdc[28692](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041289, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.194.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 18:34:49.217756 29072 init.cc:377] Logged in from keytab as kudu/127.19.194.3@KRBTEST.COM (short username kudu)
I20260806 18:34:49.218672 29072 webserver.cc:460] Webserver started at http://127.19.194.3:37105/ using document root <none> and password file <none>
I20260806 18:34:49.219273 29072 fs_manager.cc:359] Metadata directory not provided
I20260806 18:34:49.219352 29072 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 18:34:49.219516 29072 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 18:34:49.221415 29072 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786041246094603-20232-0/minicluster-data/ts-2/data/instance:
uuid: "7e0706bca9fc42cbbe19768bbf128c84"
format_stamp: "Formatted at 2026-08-06 18:34:49 on dist-test-slave-1xrh"
server_key: "7b068311a1b00aa312977d3b910c91a0"
server_key_iv: "bcfc703440229ca0d2eed54f3ef8fdad"
server_key_version: "encryptionkey@0"
I20260806 18:34:49.221908 29072 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786041246094603-20232-0/minicluster-data/ts-2/wal/instance:
uuid: "7e0706bca9fc42cbbe19768bbf128c84"
format_stamp: "Formatted at 2026-08-06 18:34:49 on dist-test-slave-1xrh"
server_key: "7b068311a1b00aa312977d3b910c91a0"
server_key_iv: "bcfc703440229ca0d2eed54f3ef8fdad"
server_key_version: "encryptionkey@0"
I20260806 18:34:49.225723 29072 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260806 18:34:49.228039 29087 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 18:34:49.229198 29072 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260806 18:34:49.229321 29072 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786041246094603-20232-0/minicluster-data/ts-2/data,/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786041246094603-20232-0/minicluster-data/ts-2/wal
uuid: "7e0706bca9fc42cbbe19768bbf128c84"
format_stamp: "Formatted at 2026-08-06 18:34:49 on dist-test-slave-1xrh"
server_key: "7b068311a1b00aa312977d3b910c91a0"
server_key_iv: "bcfc703440229ca0d2eed54f3ef8fdad"
server_key_version: "encryptionkey@0"
I20260806 18:34:49.229422 29072 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786041246094603-20232-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786041246094603-20232-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786041246094603-20232-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 18:34:49.242162 29072 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 18:34:49.244894 29072 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 18:34:49.245105 29072 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 18:34:49.245647 29072 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 18:34:49.246538 29072 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 18:34:49.246596 29072 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 18:34:49.246686 29072 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 18:34:49.246723 29072 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 18:34:49.256110 29072 rpc_server.cc:307] RPC server started. Bound to: 127.19.194.3:41979
I20260806 18:34:49.256139 29203 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.194.3:41979 every 8 connection(s)
I20260806 18:34:49.256948 29072 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786041246094603-20232-0/minicluster-data/ts-2/data/info.pb
I20260806 18:34:49.258770 20232 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJW69tn/build/debug/bin/kudu as pid 29072
I20260806 18:34:49.258894 20232 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786041246094603-20232-0/minicluster-data/ts-2/wal/instance
I20260806 18:34:49.259220 20232 external_mini_cluster.cc:1428] Setting key 512ca93b8b9a208938bd5711bb26bb8a
Aug 06 18:34:49 dist-test-slave-1xrh krb5kdc[28692](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041289, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.194.3@KRBTEST.COM for kudu/127.19.194.62@KRBTEST.COM
I20260806 18:34:49.269160 28777 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:34:49.258681 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.194.3:36215 (local address 127.19.194.62:44589)
0806 18:34:49.258861 (+ 180us) server_negotiation.cc:206] Beginning negotiation
0806 18:34:49.258868 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 18:34:49.259945 (+ 1077us) server_negotiation.cc:407] Connection header received
0806 18:34:49.260868 (+ 923us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:34:49.260871 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:34:49.260921 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:34:49.260995 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:34:49.262080 (+ 1085us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:34:49.262562 (+ 482us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:34:49.263612 (+ 1050us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:34:49.263820 (+ 208us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:34:49.263917 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:34:49.266189 (+ 2272us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:34:49.266192 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:34:49.266194 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 18:34:49.266214 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:34:49.267500 (+ 1286us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:34:49.267946 (+ 446us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:34:49.267949 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:34:49.267950 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:34:49.268043 (+ 93us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:34:49.268325 (+ 282us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:34:49.268327 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:34:49.268328 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:34:49.268484 (+ 156us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:34:49.268556 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0806 18:34:49.268949 (+ 393us) server_negotiation.cc:299] Negotiation successful
0806 18:34:49.269044 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":75}
I20260806 18:34:49.269899 29206 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:34:49.259033 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.194.62:44589 (local address 127.19.194.3:36215)
0806 18:34:49.259836 (+ 803us) negotiation.cc:107] Waiting for socket to connect
0806 18:34:49.259873 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0806 18:34:49.260690 (+ 817us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:34:49.261111 (+ 421us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:34:49.261117 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:34:49.261484 (+ 367us) client_negotiation.cc:189] Negotiated authn=SASL
0806 18:34:49.261926 (+ 442us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:34:49.261939 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:34:49.262666 (+ 727us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:34:49.262668 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:34:49.263513 (+ 845us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:34:49.263522 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:34:49.263975 (+ 453us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:34:49.263978 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:34:49.264105 (+ 127us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:34:49.264660 (+ 555us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 18:34:49.264675 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0806 18:34:49.266054 (+ 1379us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 18:34:49.267603 (+ 1549us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:34:49.267608 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:34:49.267618 (+ 10us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:34:49.267855 (+ 237us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:34:49.268123 (+ 268us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:34:49.268125 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:34:49.268126 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:34:49.268247 (+ 121us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:34:49.268570 (+ 323us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 18:34:49.268574 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 18:34:49.268762 (+ 188us) client_negotiation.cc:769] Sending connection context
0806 18:34:49.268901 (+ 139us) client_negotiation.cc:240] Negotiation successful
0806 18:34:49.269123 (+ 222us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":623,"thread_start_us":138,"threads_started":1}
I20260806 18:34:49.270957 29204 heartbeater.cc:347] Connected to a master server at 127.19.194.62:44589
I20260806 18:34:49.271224 29204 heartbeater.cc:464] Registering TS with master...
I20260806 18:34:49.271690 29204 heartbeater.cc:511] Master 127.19.194.62:44589 requested a full tablet report, sending...
I20260806 18:34:49.272786 28738 ts_manager.cc:194] Registered new tserver with Master: 7e0706bca9fc42cbbe19768bbf128c84 (127.19.194.3:41979)
I20260806 18:34:49.273277 20232 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260806 18:34:49.273535 28738 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.19.194.3@KRBTEST.COM'} at 127.19.194.3:36215
WARNING: no policy specified for oryx/127.19.194.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.19.194.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.19.194.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786041246094603-20232-0/krb5kdc/oryx_127.19.194.1.keytab.
Entry for principal oryx/127.19.194.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786041246094603-20232-0/krb5kdc/oryx_127.19.194.1.keytab.
WARNING: no policy specified for oryx/127.19.194.2@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.19.194.2@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.19.194.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786041246094603-20232-0/krb5kdc/oryx_127.19.194.2.keytab.
Entry for principal oryx/127.19.194.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786041246094603-20232-0/krb5kdc/oryx_127.19.194.2.keytab.
WARNING: no policy specified for oryx/127.19.194.3@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.19.194.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.19.194.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786041246094603-20232-0/krb5kdc/oryx_127.19.194.3.keytab.
Entry for principal oryx/127.19.194.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786041246094603-20232-0/krb5kdc/oryx_127.19.194.3.keytab.
I20260806 18:34:49.348217 20232 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJW69tn/build/debug/bin/kudu with pid 28800
I20260806 18:34:49.355360 20232 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJW69tn/build/debug/bin/kudu with pid 28935
I20260806 18:34:49.362807 20232 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJW69tn/build/debug/bin/kudu with pid 29072
I20260806 18:34:49.369611 20232 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJW69tn/build/debug/bin/kudu with pid 28705
I20260806 18:34:49.377115 20232 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJW69tn/build/debug/bin/kudu
/tmp/dist-test-taskJW69tn/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786041246094603-20232-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786041246094603-20232-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786041246094603-20232-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786041246094603-20232-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.19.194.62:44589
--webserver_interface=127.19.194.62
--webserver_port=40121
--keytab_file=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786041246094603-20232-0/krb5kdc/kudu_127.19.194.62.keytab
--principal=kudu/127.19.194.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.19.194.20:42015
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.19.194.62:44589
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786041246094603-20232-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786041246094603-20232-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786041246094603-20232-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 18:34:49.484680 29213 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 18:34:49.484962 29213 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 18:34:49.485059 29213 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 18:34:49.488888 29213 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 18:34:49.488965 29213 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 18:34:49.489001 29213 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 18:34:49.489075 29213 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 18:34:49.489111 29213 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 18:34:49.493772 29213 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.194.20:42015
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786041246094603-20232-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786041246094603-20232-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.19.194.62:44589
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786041246094603-20232-0/krb5kdc/kudu_127.19.194.62.keytab
--principal=kudu/127.19.194.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.19.194.62:44589
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786041246094603-20232-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.19.194.62
--webserver_port=40121
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29213
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786041246094603-20232-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 17:59:13 UTC on ccycloud.quasar-wvsnzx.root.comops.site
build id 4571028
I20260806 18:34:49.494751 29213 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 18:34:49.495746 29213 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 18:34:49.503917 29219 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:34:49.503923 29218 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:34:49.503940 29221 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 18:34:49.504268 29213 server_base.cc:1034] running on GCE node
I20260806 18:34:49.504838 29213 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 18:34:49.505887 29213 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 18:34:49.507052 29213 hybrid_clock.cc:648] HybridClock initialized: now 1786041289507033 us; error 31 us; skew 500 ppm
Aug 06 18:34:49 dist-test-slave-1xrh krb5kdc[28692](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041289, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.194.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 18:34:49.509771 29213 init.cc:377] Logged in from keytab as kudu/127.19.194.62@KRBTEST.COM (short username kudu)
I20260806 18:34:49.510641 29213 webserver.cc:460] Webserver started at http://127.19.194.62:40121/ using document root <none> and password file <none>
I20260806 18:34:49.511245 29213 fs_manager.cc:359] Metadata directory not provided
I20260806 18:34:49.511360 29213 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 18:34:49.514966 29213 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.003s sys 0.000s
I20260806 18:34:49.517017 29228 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 18:34:49.518121 29213 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260806 18:34:49.518262 29213 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786041246094603-20232-0/minicluster-data/master-0/data,/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786041246094603-20232-0/minicluster-data/master-0/wal
uuid: "189709350f1345d8a5ecbcfd6d1c55f2"
format_stamp: "Formatted at 2026-08-06 18:34:48 on dist-test-slave-1xrh"
server_key: "ec57af6bdc7ef463964fbaca1ee22045"
server_key_iv: "870fe58f6097e3b9c83fbb94f87dd50c"
server_key_version: "encryptionkey@0"
I20260806 18:34:49.518788 29213 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786041246094603-20232-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786041246094603-20232-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786041246094603-20232-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 18:34:49.551496 29213 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 18:34:49.554203 29213 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 18:34:49.554396 29213 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 18:34:49.561862 29213 rpc_server.cc:307] RPC server started. Bound to: 127.19.194.62:44589
I20260806 18:34:49.561889 29281 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.194.62:44589 every 8 connection(s)
I20260806 18:34:49.562875 29213 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786041246094603-20232-0/minicluster-data/master-0/data/info.pb
I20260806 18:34:49.563655 20232 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJW69tn/build/debug/bin/kudu as pid 29213
I20260806 18:34:49.564199 20232 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJW69tn/build/debug/bin/kudu
/tmp/dist-test-taskJW69tn/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786041246094603-20232-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786041246094603-20232-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786041246094603-20232-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786041246094603-20232-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.19.194.1:44719
--local_ip_for_outbound_sockets=127.19.194.1
--tserver_master_addrs=127.19.194.62:44589
--webserver_port=40065
--webserver_interface=127.19.194.1
--keytab_file=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786041246094603-20232-0/krb5kdc/kudu_127.19.194.1.keytab
--principal=kudu/127.19.194.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.19.194.20:42015
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.19.194.1
--keytab_file=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786041246094603-20232-0/krb5kdc/oryx_127.19.194.1.keytab with env {KRB5CCNAME=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786041246094603-20232-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786041246094603-20232-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786041246094603-20232-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260806 18:34:49.568109 29282 sys_catalog.cc:263] Verifying existing consensus state
I20260806 18:34:49.569751 29282 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 189709350f1345d8a5ecbcfd6d1c55f2: Bootstrap starting.
I20260806 18:34:49.586778 29282 log.cc:824] T 00000000000000000000000000000000 P 189709350f1345d8a5ecbcfd6d1c55f2: Log is configured to *not* fsync() on all Append() calls
I20260806 18:34:49.593130 29282 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 189709350f1345d8a5ecbcfd6d1c55f2: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260806 18:34:49.593503 29282 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 189709350f1345d8a5ecbcfd6d1c55f2: Bootstrap complete.
I20260806 18:34:49.597801 29282 raft_consensus.cc:348] T 00000000000000000000000000000000 P 189709350f1345d8a5ecbcfd6d1c55f2 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "189709350f1345d8a5ecbcfd6d1c55f2" member_type: VOTER last_known_addr { host: "127.19.194.62" port: 44589 } }
I20260806 18:34:49.598187 29282 raft_consensus.cc:729] T 00000000000000000000000000000000 P 189709350f1345d8a5ecbcfd6d1c55f2 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 189709350f1345d8a5ecbcfd6d1c55f2, State: Initialized, Role: FOLLOWER
I20260806 18:34:49.598719 29282 consensus_queue.cc:260] T 00000000000000000000000000000000 P 189709350f1345d8a5ecbcfd6d1c55f2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "189709350f1345d8a5ecbcfd6d1c55f2" member_type: VOTER last_known_addr { host: "127.19.194.62" port: 44589 } }
I20260806 18:34:49.598837 29282 raft_consensus.cc:388] T 00000000000000000000000000000000 P 189709350f1345d8a5ecbcfd6d1c55f2 [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 18:34:49.598922 29282 raft_consensus.cc:482] T 00000000000000000000000000000000 P 189709350f1345d8a5ecbcfd6d1c55f2 [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 18:34:49.599038 29282 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 189709350f1345d8a5ecbcfd6d1c55f2 [term 1 FOLLOWER]: Advancing to term 2
I20260806 18:34:49.600960 29282 raft_consensus.cc:504] T 00000000000000000000000000000000 P 189709350f1345d8a5ecbcfd6d1c55f2 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "189709350f1345d8a5ecbcfd6d1c55f2" member_type: VOTER last_known_addr { host: "127.19.194.62" port: 44589 } }
I20260806 18:34:49.601322 29282 leader_election.cc:302] T 00000000000000000000000000000000 P 189709350f1345d8a5ecbcfd6d1c55f2 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 189709350f1345d8a5ecbcfd6d1c55f2; no voters:
I20260806 18:34:49.601632 29282 leader_election.cc:288] T 00000000000000000000000000000000 P 189709350f1345d8a5ecbcfd6d1c55f2 [CANDIDATE]: Term 2 election: Requested vote from peers
I20260806 18:34:49.601788 29286 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 189709350f1345d8a5ecbcfd6d1c55f2 [term 2 FOLLOWER]: Leader election won for term 2
I20260806 18:34:49.602044 29286 raft_consensus.cc:686] T 00000000000000000000000000000000 P 189709350f1345d8a5ecbcfd6d1c55f2 [term 2 LEADER]: Becoming Leader. State: Replica: 189709350f1345d8a5ecbcfd6d1c55f2, State: Running, Role: LEADER
I20260806 18:34:49.602340 29286 consensus_queue.cc:237] T 00000000000000000000000000000000 P 189709350f1345d8a5ecbcfd6d1c55f2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "189709350f1345d8a5ecbcfd6d1c55f2" member_type: VOTER last_known_addr { host: "127.19.194.62" port: 44589 } }
I20260806 18:34:49.602825 29282 sys_catalog.cc:564] T 00000000000000000000000000000000 P 189709350f1345d8a5ecbcfd6d1c55f2 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 18:34:49.603682 29287 sys_catalog.cc:455] T 00000000000000000000000000000000 P 189709350f1345d8a5ecbcfd6d1c55f2 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "189709350f1345d8a5ecbcfd6d1c55f2" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "189709350f1345d8a5ecbcfd6d1c55f2" member_type: VOTER last_known_addr { host: "127.19.194.62" port: 44589 } } }
I20260806 18:34:49.603829 29287 sys_catalog.cc:458] T 00000000000000000000000000000000 P 189709350f1345d8a5ecbcfd6d1c55f2 [sys.catalog]: This master's current role is: LEADER
I20260806 18:34:49.604494 29288 sys_catalog.cc:455] T 00000000000000000000000000000000 P 189709350f1345d8a5ecbcfd6d1c55f2 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 189709350f1345d8a5ecbcfd6d1c55f2. Latest consensus state: current_term: 2 leader_uuid: "189709350f1345d8a5ecbcfd6d1c55f2" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "189709350f1345d8a5ecbcfd6d1c55f2" member_type: VOTER last_known_addr { host: "127.19.194.62" port: 44589 } } }
I20260806 18:34:49.604592 29288 sys_catalog.cc:458] T 00000000000000000000000000000000 P 189709350f1345d8a5ecbcfd6d1c55f2 [sys.catalog]: This master's current role is: LEADER
I20260806 18:34:49.604990 29295 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 18:34:49.608321 29295 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 18:34:49.609031 29295 catalog_manager.cc:1259] Loaded cluster ID: ad68f8ee2d84469caadc916bd02be3df
I20260806 18:34:49.609090 29295 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 18:34:49.611279 29295 catalog_manager.cc:1504] Loading token signing keys...
I20260806 18:34:49.611886 29295 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 189709350f1345d8a5ecbcfd6d1c55f2: Loaded TSK: 0
I20260806 18:34:49.612993 29295 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260806 18:34:49.678378 29284 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 18:34:49.678601 29284 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 18:34:49.678639 29284 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 18:34:49.682511 29284 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 18:34:49.682562 29284 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 18:34:49.682616 29284 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.19.194.1
I20260806 18:34:49.687650 29284 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.194.20:42015
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786041246094603-20232-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786041246094603-20232-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786041246094603-20232-0/krb5kdc/oryx_127.19.194.1.keytab
--principal=oryx/127.19.194.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.19.194.1:44719
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786041246094603-20232-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.19.194.1
--webserver_port=40065
--tserver_master_addrs=127.19.194.62:44589
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29284
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.19.194.1
--log_dir=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786041246094603-20232-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 17:59:13 UTC on ccycloud.quasar-wvsnzx.root.comops.site
build id 4571028
I20260806 18:34:49.688567 29284 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 18:34:49.689507 29284 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 18:34:49.698983 29312 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:34:49.698989 29310 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 18:34:49.699173 29284 server_base.cc:1034] running on GCE node
W20260806 18:34:49.699290 29309 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 18:34:49.699671 29284 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 18:34:49.700430 29284 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 18:34:49.701592 29284 hybrid_clock.cc:648] HybridClock initialized: now 1786041289701574 us; error 33 us; skew 500 ppm
Aug 06 18:34:49 dist-test-slave-1xrh krb5kdc[28692](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041289, etypes {rep=17 tkt=17 ses=17}, oryx/127.19.194.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 18:34:49.704836 29284 init.cc:377] Logged in from keytab as oryx/127.19.194.1@KRBTEST.COM (short username oryx)
I20260806 18:34:49.705988 29284 webserver.cc:460] Webserver started at http://127.19.194.1:40065/ using document root <none> and password file <none>
I20260806 18:34:49.706661 29284 fs_manager.cc:359] Metadata directory not provided
I20260806 18:34:49.706763 29284 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 18:34:49.711716 29284 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.003s sys 0.000s
I20260806 18:34:49.714306 29319 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 18:34:49.715818 29284 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260806 18:34:49.715956 29284 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786041246094603-20232-0/minicluster-data/ts-0/data,/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786041246094603-20232-0/minicluster-data/ts-0/wal
uuid: "1c30e79bd0454f94b1b6fe14f5c23888"
format_stamp: "Formatted at 2026-08-06 18:34:48 on dist-test-slave-1xrh"
server_key: "b2a1925ee37bfff885c3d4e0ce6936de"
server_key_iv: "68870f74d74d6581a080639467947330"
server_key_version: "encryptionkey@0"
I20260806 18:34:49.716647 29284 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786041246094603-20232-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786041246094603-20232-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786041246094603-20232-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 18:34:49.761897 29284 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 18:34:49.764858 29284 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 18:34:49.765017 29284 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 18:34:49.765588 29284 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 18:34:49.766371 29284 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 18:34:49.766422 29284 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 18:34:49.766463 29284 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 18:34:49.766481 29284 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 18:34:49.775527 29284 rpc_server.cc:307] RPC server started. Bound to: 127.19.194.1:44719
I20260806 18:34:49.775571 29434 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.194.1:44719 every 8 connection(s)
I20260806 18:34:49.776548 29284 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786041246094603-20232-0/minicluster-data/ts-0/data/info.pb
I20260806 18:34:49.781421 20232 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJW69tn/build/debug/bin/kudu as pid 29284
I20260806 18:34:49.781877 20232 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJW69tn/build/debug/bin/kudu
/tmp/dist-test-taskJW69tn/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786041246094603-20232-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786041246094603-20232-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786041246094603-20232-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786041246094603-20232-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.19.194.2:33567
--local_ip_for_outbound_sockets=127.19.194.2
--tserver_master_addrs=127.19.194.62:44589
--webserver_port=34001
--webserver_interface=127.19.194.2
--keytab_file=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786041246094603-20232-0/krb5kdc/kudu_127.19.194.2.keytab
--principal=kudu/127.19.194.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.19.194.20:42015
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.19.194.2
--keytab_file=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786041246094603-20232-0/krb5kdc/oryx_127.19.194.2.keytab with env {KRB5CCNAME=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786041246094603-20232-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786041246094603-20232-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786041246094603-20232-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 06 18:34:49 dist-test-slave-1xrh krb5kdc[28692](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.19.194.1@KRBTEST.COM for oryx/127.19.194.62@KRBTEST.COM, Server not found in Kerberos database
Aug 06 18:34:49 dist-test-slave-1xrh krb5kdc[28692](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.19.194.1@KRBTEST.COM for oryx/127.19.194.62@KRBTEST.COM, Server not found in Kerberos database
W20260806 18:34:49.790249 29438 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 18:34:49.778575 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.194.1:53717 (local address 127.19.194.62:44589)
0806 18:34:49.779049 (+ 474us) server_negotiation.cc:206] Beginning negotiation
0806 18:34:49.779071 (+ 22us) server_negotiation.cc:399] Waiting for connection header
0806 18:34:49.779332 (+ 261us) server_negotiation.cc:407] Connection header received
0806 18:34:49.780788 (+ 1456us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:34:49.780798 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:34:49.781086 (+ 288us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:34:49.781414 (+ 328us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:34:49.782603 (+ 1189us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:34:49.783987 (+ 1384us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:34:49.785123 (+ 1136us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:34:49.785617 (+ 494us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:34:49.785798 (+ 181us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:34:49.789516 (+ 3718us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.19.194.1:53717: BlockingRecv error: recv got EOF from 127.19.194.1:53717 (error 108)
Metrics: {"server-negotiator.queue_time_us":315,"thread_start_us":144,"threads_started":1}
I20260806 18:34:49.790266 29439 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:34:49.778635 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.194.62:44589 (local address 127.19.194.1:53717)
0806 18:34:49.779166 (+ 531us) negotiation.cc:107] Waiting for socket to connect
0806 18:34:49.779217 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0806 18:34:49.780432 (+ 1215us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:34:49.781485 (+ 1053us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:34:49.781493 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:34:49.781910 (+ 417us) client_negotiation.cc:189] Negotiated authn=SASL
0806 18:34:49.782428 (+ 518us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:34:49.782442 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:34:49.784143 (+ 1701us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:34:49.784147 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:34:49.784990 (+ 843us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:34:49.785000 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:34:49.785791 (+ 791us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:34:49.785795 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:34:49.785991 (+ 196us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:34:49.786640 (+ 649us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 18:34:49.786656 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0806 18:34:49.789303 (+ 2647us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.19.194.62:44589: Server oryx/127.19.194.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":282,"thread_start_us":148,"threads_started":1}
W20260806 18:34:49.791023 29435 heartbeater.cc:644] Failed to heartbeat to 127.19.194.62:44589 (0 consecutive failures): Not authorized: Failed to ping master at 127.19.194.62:44589: Client connection negotiation failed: client connection to 127.19.194.62:44589: Server oryx/127.19.194.62@KRBTEST.COM not found in Kerberos database
Aug 06 18:34:49 dist-test-slave-1xrh krb5kdc[28692](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.19.194.1@KRBTEST.COM for oryx/127.19.194.62@KRBTEST.COM, Server not found in Kerberos database
Aug 06 18:34:49 dist-test-slave-1xrh krb5kdc[28692](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.19.194.1@KRBTEST.COM for oryx/127.19.194.62@KRBTEST.COM, Server not found in Kerberos database
I20260806 18:34:49.796458 29439 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:34:49.791498 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.194.62:44589 (local address 127.19.194.1:33637)
0806 18:34:49.791590 (+ 92us) negotiation.cc:107] Waiting for socket to connect
0806 18:34:49.791604 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0806 18:34:49.791706 (+ 102us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:34:49.791980 (+ 274us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:34:49.791982 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:34:49.792140 (+ 158us) client_negotiation.cc:189] Negotiated authn=SASL
0806 18:34:49.792226 (+ 86us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:34:49.792231 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:34:49.792940 (+ 709us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:34:49.792945 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:34:49.793455 (+ 510us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:34:49.793479 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:34:49.793925 (+ 446us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:34:49.793928 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:34:49.794019 (+ 91us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:34:49.794507 (+ 488us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 18:34:49.794518 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0806 18:34:49.796299 (+ 1781us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.19.194.62:44589: Server oryx/127.19.194.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":27}
W20260806 18:34:49.796844 29438 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 18:34:49.791618 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.194.1:33637 (local address 127.19.194.62:44589)
0806 18:34:49.791769 (+ 151us) server_negotiation.cc:206] Beginning negotiation
0806 18:34:49.791775 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 18:34:49.791786 (+ 11us) server_negotiation.cc:407] Connection header received
0806 18:34:49.791822 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:34:49.791824 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:34:49.791859 (+ 35us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:34:49.791971 (+ 112us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:34:49.792343 (+ 372us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:34:49.792830 (+ 487us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:34:49.793600 (+ 770us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:34:49.793817 (+ 217us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:34:49.793922 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:34:49.796438 (+ 2516us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.19.194.1:33637: BlockingRecv error: recv got EOF from 127.19.194.1:33637 (error 108)
Metrics: {"server-negotiator.queue_time_us":60}
Aug 06 18:34:49 dist-test-slave-1xrh krb5kdc[28692](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.19.194.1@KRBTEST.COM for oryx/127.19.194.62@KRBTEST.COM, Server not found in Kerberos database
Aug 06 18:34:49 dist-test-slave-1xrh krb5kdc[28692](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.19.194.1@KRBTEST.COM for oryx/127.19.194.62@KRBTEST.COM, Server not found in Kerberos database
I20260806 18:34:49.802026 29439 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:34:49.797047 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.194.62:44589 (local address 127.19.194.1:46009)
0806 18:34:49.797306 (+ 259us) negotiation.cc:107] Waiting for socket to connect
0806 18:34:49.797321 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0806 18:34:49.797427 (+ 106us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:34:49.797680 (+ 253us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:34:49.797683 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:34:49.797834 (+ 151us) client_negotiation.cc:189] Negotiated authn=SASL
0806 18:34:49.797897 (+ 63us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:34:49.797902 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:34:49.798585 (+ 683us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:34:49.798589 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:34:49.799112 (+ 523us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:34:49.799168 (+ 56us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:34:49.799614 (+ 446us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:34:49.799617 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:34:49.799699 (+ 82us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:34:49.800178 (+ 479us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 18:34:49.800189 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0806 18:34:49.801898 (+ 1709us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.19.194.62:44589: Server oryx/127.19.194.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":47}
W20260806 18:34:49.802119 29438 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 18:34:49.797181 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.194.1:46009 (local address 127.19.194.62:44589)
0806 18:34:49.797288 (+ 107us) server_negotiation.cc:206] Beginning negotiation
0806 18:34:49.797292 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 18:34:49.797389 (+ 97us) server_negotiation.cc:407] Connection header received
0806 18:34:49.797533 (+ 144us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:34:49.797536 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:34:49.797581 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:34:49.797663 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:34:49.798014 (+ 351us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:34:49.798473 (+ 459us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:34:49.799290 (+ 817us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:34:49.799497 (+ 207us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:34:49.799601 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:34:49.802013 (+ 2412us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.19.194.1:46009: BlockingRecv error: recv got EOF from 127.19.194.1:46009 (error 108)
Metrics: {"server-negotiator.queue_time_us":43}
W20260806 18:34:49.802436 29435 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260806 18:34:49.894515 29440 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 18:34:49.894800 29440 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 18:34:49.894889 29440 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 18:34:49.898972 29440 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 18:34:49.899070 29440 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 18:34:49.899214 29440 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.19.194.2
I20260806 18:34:49.904726 29440 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.194.20:42015
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786041246094603-20232-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786041246094603-20232-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786041246094603-20232-0/krb5kdc/oryx_127.19.194.2.keytab
--principal=oryx/127.19.194.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.19.194.2:33567
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786041246094603-20232-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.19.194.2
--webserver_port=34001
--tserver_master_addrs=127.19.194.62:44589
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29440
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.19.194.2
--log_dir=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786041246094603-20232-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 17:59:13 UTC on ccycloud.quasar-wvsnzx.root.comops.site
build id 4571028
I20260806 18:34:49.905716 29440 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 18:34:49.906725 29440 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 18:34:49.916076 29448 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:34:49.916076 29446 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 18:34:49.916290 29440 server_base.cc:1034] running on GCE node
W20260806 18:34:49.916182 29445 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 18:34:49.916736 29440 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 18:34:49.917377 29440 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 18:34:49.918584 29440 hybrid_clock.cc:648] HybridClock initialized: now 1786041289918562 us; error 31 us; skew 500 ppm
Aug 06 18:34:49 dist-test-slave-1xrh krb5kdc[28692](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041289, etypes {rep=17 tkt=17 ses=17}, oryx/127.19.194.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 18:34:49.921252 29440 init.cc:377] Logged in from keytab as oryx/127.19.194.2@KRBTEST.COM (short username oryx)
I20260806 18:34:49.922223 29440 webserver.cc:460] Webserver started at http://127.19.194.2:34001/ using document root <none> and password file <none>
I20260806 18:34:49.922920 29440 fs_manager.cc:359] Metadata directory not provided
I20260806 18:34:49.923032 29440 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 18:34:49.926719 29440 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.002s sys 0.001s
I20260806 18:34:49.928754 29456 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 18:34:49.929901 29440 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260806 18:34:49.930045 29440 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786041246094603-20232-0/minicluster-data/ts-1/data,/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786041246094603-20232-0/minicluster-data/ts-1/wal
uuid: "e6f6733b4f044979bb83e56cb63cbcb6"
format_stamp: "Formatted at 2026-08-06 18:34:48 on dist-test-slave-1xrh"
server_key: "ef681e181beb8b9b22e5068ee96f7111"
server_key_iv: "76d6230dca250e32308b2dba34761b9d"
server_key_version: "encryptionkey@0"
I20260806 18:34:49.930575 29440 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786041246094603-20232-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786041246094603-20232-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786041246094603-20232-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 18:34:49.962582 29440 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 18:34:49.965577 29440 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 18:34:49.965788 29440 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 18:34:49.966378 29440 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 18:34:49.967222 29440 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 18:34:49.967298 29440 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 18:34:49.967371 29440 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 18:34:49.967407 29440 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 18:34:49.976523 29440 rpc_server.cc:307] RPC server started. Bound to: 127.19.194.2:33567
I20260806 18:34:49.976552 29572 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.194.2:33567 every 8 connection(s)
I20260806 18:34:49.977484 29440 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786041246094603-20232-0/minicluster-data/ts-1/data/info.pb
I20260806 18:34:49.979620 20232 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJW69tn/build/debug/bin/kudu as pid 29440
I20260806 18:34:49.980094 20232 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJW69tn/build/debug/bin/kudu
/tmp/dist-test-taskJW69tn/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786041246094603-20232-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786041246094603-20232-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786041246094603-20232-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786041246094603-20232-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.19.194.3:41979
--local_ip_for_outbound_sockets=127.19.194.3
--tserver_master_addrs=127.19.194.62:44589
--webserver_port=37105
--webserver_interface=127.19.194.3
--keytab_file=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786041246094603-20232-0/krb5kdc/kudu_127.19.194.3.keytab
--principal=kudu/127.19.194.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.19.194.20:42015
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.19.194.3
--keytab_file=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786041246094603-20232-0/krb5kdc/oryx_127.19.194.3.keytab with env {KRB5CCNAME=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786041246094603-20232-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786041246094603-20232-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786041246094603-20232-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 06 18:34:49 dist-test-slave-1xrh krb5kdc[28692](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.19.194.2@KRBTEST.COM for oryx/127.19.194.62@KRBTEST.COM, Server not found in Kerberos database
Aug 06 18:34:49 dist-test-slave-1xrh krb5kdc[28692](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.19.194.2@KRBTEST.COM for oryx/127.19.194.62@KRBTEST.COM, Server not found in Kerberos database
W20260806 18:34:49.990200 29438 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 18:34:49.979295 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.194.2:39515 (local address 127.19.194.62:44589)
0806 18:34:49.979473 (+ 178us) server_negotiation.cc:206] Beginning negotiation
0806 18:34:49.979478 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 18:34:49.980680 (+ 1202us) server_negotiation.cc:407] Connection header received
0806 18:34:49.981551 (+ 871us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:34:49.981557 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:34:49.981632 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:34:49.981743 (+ 111us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:34:49.983381 (+ 1638us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:34:49.983932 (+ 551us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:34:49.985849 (+ 1917us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:34:49.986073 (+ 224us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:34:49.986155 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:34:49.990070 (+ 3915us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.19.194.2:39515: BlockingRecv error: recv got EOF from 127.19.194.2:39515 (error 108)
Metrics: {"server-negotiator.queue_time_us":66}
I20260806 18:34:49.990808 29575 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:34:49.979696 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.194.62:44589 (local address 127.19.194.2:39515)
0806 18:34:49.980502 (+ 806us) negotiation.cc:107] Waiting for socket to connect
0806 18:34:49.980555 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0806 18:34:49.981334 (+ 779us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:34:49.981962 (+ 628us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:34:49.981974 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:34:49.982498 (+ 524us) client_negotiation.cc:189] Negotiated authn=SASL
0806 18:34:49.983222 (+ 724us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:34:49.983245 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:34:49.984076 (+ 831us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:34:49.984083 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:34:49.985701 (+ 1618us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:34:49.985716 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:34:49.986250 (+ 534us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:34:49.986257 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:34:49.986510 (+ 253us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:34:49.987235 (+ 725us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 18:34:49.987262 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0806 18:34:49.989935 (+ 2673us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.19.194.62:44589: Server oryx/127.19.194.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":578,"thread_start_us":169,"threads_started":1}
W20260806 18:34:49.991408 29573 heartbeater.cc:644] Failed to heartbeat to 127.19.194.62:44589 (0 consecutive failures): Not authorized: Failed to ping master at 127.19.194.62:44589: Client connection negotiation failed: client connection to 127.19.194.62:44589: Server oryx/127.19.194.62@KRBTEST.COM not found in Kerberos database
Aug 06 18:34:49 dist-test-slave-1xrh krb5kdc[28692](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.19.194.2@KRBTEST.COM for oryx/127.19.194.62@KRBTEST.COM, Server not found in Kerberos database
Aug 06 18:34:49 dist-test-slave-1xrh krb5kdc[28692](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.19.194.2@KRBTEST.COM for oryx/127.19.194.62@KRBTEST.COM, Server not found in Kerberos database
I20260806 18:34:49.997257 29575 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:34:49.991902 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.194.62:44589 (local address 127.19.194.2:49925)
0806 18:34:49.992021 (+ 119us) negotiation.cc:107] Waiting for socket to connect
0806 18:34:49.992043 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0806 18:34:49.992117 (+ 74us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:34:49.992392 (+ 275us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:34:49.992394 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:34:49.992518 (+ 124us) client_negotiation.cc:189] Negotiated authn=SASL
0806 18:34:49.992606 (+ 88us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:34:49.992611 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:34:49.993585 (+ 974us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:34:49.993587 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:34:49.994096 (+ 509us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:34:49.994109 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:34:49.994648 (+ 539us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:34:49.994650 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:34:49.994718 (+ 68us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:34:49.995315 (+ 597us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 18:34:49.995327 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0806 18:34:49.997171 (+ 1844us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.19.194.62:44589: Server oryx/127.19.194.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":52}
W20260806 18:34:49.997443 29438 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 18:34:49.992009 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.194.2:49925 (local address 127.19.194.62:44589)
0806 18:34:49.992155 (+ 146us) server_negotiation.cc:206] Beginning negotiation
0806 18:34:49.992161 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 18:34:49.992176 (+ 15us) server_negotiation.cc:407] Connection header received
0806 18:34:49.992218 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:34:49.992222 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:34:49.992278 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:34:49.992374 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:34:49.992713 (+ 339us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:34:49.993482 (+ 769us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:34:49.994208 (+ 726us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:34:49.994542 (+ 334us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:34:49.994637 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:34:49.997322 (+ 2685us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.19.194.2:49925: BlockingRecv error: recv got EOF from 127.19.194.2:49925 (error 108)
Metrics: {"server-negotiator.queue_time_us":59}
Aug 06 18:34:49 dist-test-slave-1xrh krb5kdc[28692](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.19.194.2@KRBTEST.COM for oryx/127.19.194.62@KRBTEST.COM, Server not found in Kerberos database
Aug 06 18:34:49 dist-test-slave-1xrh krb5kdc[28692](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.19.194.2@KRBTEST.COM for oryx/127.19.194.62@KRBTEST.COM, Server not found in Kerberos database
I20260806 18:34:50.002982 29575 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:34:49.997892 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.194.62:44589 (local address 127.19.194.2:50049)
0806 18:34:49.997979 (+ 87us) negotiation.cc:107] Waiting for socket to connect
0806 18:34:49.997995 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0806 18:34:49.998070 (+ 75us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:34:49.998296 (+ 226us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:34:49.998298 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:34:49.998413 (+ 115us) client_negotiation.cc:189] Negotiated authn=SASL
0806 18:34:49.998476 (+ 63us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:34:49.998480 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:34:49.999535 (+ 1055us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:34:49.999537 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:34:50.000039 (+ 502us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:34:50.000047 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:34:50.000599 (+ 552us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:34:50.000601 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:34:50.000657 (+ 56us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:34:50.001120 (+ 463us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 18:34:50.001130 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0806 18:34:50.002891 (+ 1761us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.19.194.62:44589: Server oryx/127.19.194.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":26}
W20260806 18:34:50.003194 29438 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 18:34:49.997966 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.194.2:50049 (local address 127.19.194.62:44589)
0806 18:34:49.998061 (+ 95us) server_negotiation.cc:206] Beginning negotiation
0806 18:34:49.998064 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 18:34:49.998077 (+ 13us) server_negotiation.cc:407] Connection header received
0806 18:34:49.998156 (+ 79us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:34:49.998159 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:34:49.998196 (+ 37us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:34:49.998272 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:34:49.998589 (+ 317us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:34:49.999427 (+ 838us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:34:50.000155 (+ 728us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:34:50.000495 (+ 340us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:34:50.000589 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:34:50.003051 (+ 2462us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.19.194.2:50049: BlockingRecv error: recv got EOF from 127.19.194.2:50049 (error 108)
Metrics: {"server-negotiator.queue_time_us":39}
W20260806 18:34:50.003332 29573 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260806 18:34:50.096172 29576 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 18:34:50.096410 29576 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 18:34:50.096454 29576 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 18:34:50.100472 29576 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 18:34:50.100526 29576 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 18:34:50.100632 29576 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.19.194.3
I20260806 18:34:50.106145 29576 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.194.20:42015
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786041246094603-20232-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786041246094603-20232-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786041246094603-20232-0/krb5kdc/oryx_127.19.194.3.keytab
--principal=oryx/127.19.194.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.19.194.3:41979
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786041246094603-20232-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.19.194.3
--webserver_port=37105
--tserver_master_addrs=127.19.194.62:44589
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29576
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.19.194.3
--log_dir=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786041246094603-20232-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 17:59:13 UTC on ccycloud.quasar-wvsnzx.root.comops.site
build id 4571028
I20260806 18:34:50.107188 29576 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 18:34:50.108172 29576 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 18:34:50.117444 29584 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:34:50.117476 29581 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:34:50.117457 29582 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 18:34:50.117748 29576 server_base.cc:1034] running on GCE node
I20260806 18:34:50.118221 29576 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 18:34:50.118810 29576 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 18:34:50.119992 29576 hybrid_clock.cc:648] HybridClock initialized: now 1786041290119939 us; error 64 us; skew 500 ppm
Aug 06 18:34:50 dist-test-slave-1xrh krb5kdc[28692](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041290, etypes {rep=17 tkt=17 ses=17}, oryx/127.19.194.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 18:34:50.122753 29576 init.cc:377] Logged in from keytab as oryx/127.19.194.3@KRBTEST.COM (short username oryx)
I20260806 18:34:50.123626 29576 webserver.cc:460] Webserver started at http://127.19.194.3:37105/ using document root <none> and password file <none>
I20260806 18:34:50.124176 29576 fs_manager.cc:359] Metadata directory not provided
I20260806 18:34:50.124291 29576 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 18:34:50.128000 29576 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.003s sys 0.000s
I20260806 18:34:50.129941 29591 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 18:34:50.131099 29576 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260806 18:34:50.131283 29576 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786041246094603-20232-0/minicluster-data/ts-2/data,/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786041246094603-20232-0/minicluster-data/ts-2/wal
uuid: "7e0706bca9fc42cbbe19768bbf128c84"
format_stamp: "Formatted at 2026-08-06 18:34:49 on dist-test-slave-1xrh"
server_key: "7b068311a1b00aa312977d3b910c91a0"
server_key_iv: "bcfc703440229ca0d2eed54f3ef8fdad"
server_key_version: "encryptionkey@0"
I20260806 18:34:50.131794 29576 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786041246094603-20232-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786041246094603-20232-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786041246094603-20232-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 18:34:50.141801 29576 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 18:34:50.144495 29576 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 18:34:50.144697 29576 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 18:34:50.145241 29576 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 18:34:50.146005 29576 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 18:34:50.146088 29576 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 18:34:50.146178 29576 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 18:34:50.146229 29576 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 18:34:50.155499 29576 rpc_server.cc:307] RPC server started. Bound to: 127.19.194.3:41979
I20260806 18:34:50.155535 29707 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.194.3:41979 every 8 connection(s)
I20260806 18:34:50.156448 29576 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786041246094603-20232-0/minicluster-data/ts-2/data/info.pb
I20260806 18:34:50.157519 20232 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJW69tn/build/debug/bin/kudu as pid 29576
Aug 06 18:34:50 dist-test-slave-1xrh krb5kdc[28692](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.19.194.3@KRBTEST.COM for oryx/127.19.194.62@KRBTEST.COM, Server not found in Kerberos database
I20260806 18:34:50.168133 29438 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:34:50.158301 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39850 (local address 127.19.194.62:44589)
0806 18:34:50.158446 (+ 145us) server_negotiation.cc:206] Beginning negotiation
0806 18:34:50.158451 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 18:34:50.158829 (+ 378us) server_negotiation.cc:407] Connection header received
0806 18:34:50.159005 (+ 176us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:34:50.159008 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:34:50.159072 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:34:50.159179 (+ 107us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:34:50.159937 (+ 758us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:34:50.160717 (+ 780us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:34:50.161666 (+ 949us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:34:50.162002 (+ 336us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:34:50.162120 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:34:50.163712 (+ 1592us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:34:50.163718 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:34:50.163730 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 18:34:50.163756 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:34:50.165745 (+ 1989us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:34:50.166216 (+ 471us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:34:50.166223 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:34:50.166229 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:34:50.166385 (+ 156us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:34:50.166719 (+ 334us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:34:50.166723 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:34:50.166724 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:34:50.167199 (+ 475us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:34:50.167410 (+ 211us) server_negotiation.cc:1035] Waiting for connection context
0806 18:34:50.167698 (+ 288us) server_negotiation.cc:299] Negotiation successful
0806 18:34:50.167933 (+ 235us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
Aug 06 18:34:50 dist-test-slave-1xrh krb5kdc[28692](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.19.194.3@KRBTEST.COM for oryx/127.19.194.62@KRBTEST.COM, Server not found in Kerberos database
W20260806 18:34:50.169485 29711 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 18:34:50.158660 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.194.3:58125 (local address 127.19.194.62:44589)
0806 18:34:50.158994 (+ 334us) server_negotiation.cc:206] Beginning negotiation
0806 18:34:50.158999 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 18:34:50.160497 (+ 1498us) server_negotiation.cc:407] Connection header received
0806 18:34:50.161416 (+ 919us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:34:50.161419 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:34:50.161468 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:34:50.161583 (+ 115us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:34:50.163012 (+ 1429us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:34:50.163828 (+ 816us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:34:50.165530 (+ 1702us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:34:50.165753 (+ 223us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:34:50.165864 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:34:50.169374 (+ 3510us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.19.194.3:58125: BlockingRecv error: recv got EOF from 127.19.194.3:58125 (error 108)
Metrics: {"server-negotiator.queue_time_us":248,"thread_start_us":106,"threads_started":1}
I20260806 18:34:50.170162 29712 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:34:50.158749 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.194.62:44589 (local address 127.19.194.3:58125)
0806 18:34:50.160343 (+ 1594us) negotiation.cc:107] Waiting for socket to connect
0806 18:34:50.160393 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0806 18:34:50.161223 (+ 830us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:34:50.161782 (+ 559us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:34:50.161790 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:34:50.162287 (+ 497us) client_negotiation.cc:189] Negotiated authn=SASL
0806 18:34:50.162865 (+ 578us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:34:50.162880 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:34:50.163958 (+ 1078us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:34:50.163963 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:34:50.165386 (+ 1423us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:34:50.165398 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:34:50.165924 (+ 526us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:34:50.165927 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:34:50.166065 (+ 138us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:34:50.166621 (+ 556us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 18:34:50.166636 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0806 18:34:50.169278 (+ 2642us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.19.194.62:44589: Server oryx/127.19.194.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":1389,"thread_start_us":125,"threads_started":1}
W20260806 18:34:50.170706 29708 heartbeater.cc:644] Failed to heartbeat to 127.19.194.62:44589 (0 consecutive failures): Not authorized: Failed to ping master at 127.19.194.62:44589: Client connection negotiation failed: client connection to 127.19.194.62:44589: Server oryx/127.19.194.62@KRBTEST.COM not found in Kerberos database
Aug 06 18:34:50 dist-test-slave-1xrh krb5kdc[28692](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.19.194.3@KRBTEST.COM for oryx/127.19.194.62@KRBTEST.COM, Server not found in Kerberos database
Aug 06 18:34:50 dist-test-slave-1xrh krb5kdc[28692](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.19.194.3@KRBTEST.COM for oryx/127.19.194.62@KRBTEST.COM, Server not found in Kerberos database
I20260806 18:34:50.175655 29712 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:34:50.171245 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.194.62:44589 (local address 127.19.194.3:36369)
0806 18:34:50.171362 (+ 117us) negotiation.cc:107] Waiting for socket to connect
0806 18:34:50.171373 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0806 18:34:50.171433 (+ 60us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:34:50.171590 (+ 157us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:34:50.171591 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:34:50.171708 (+ 117us) client_negotiation.cc:189] Negotiated authn=SASL
0806 18:34:50.171789 (+ 81us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:34:50.171793 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:34:50.172408 (+ 615us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:34:50.172410 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:34:50.172904 (+ 494us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:34:50.172911 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:34:50.173319 (+ 408us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:34:50.173322 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:34:50.173385 (+ 63us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:34:50.173870 (+ 485us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 18:34:50.173886 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0806 18:34:50.175576 (+ 1690us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.19.194.62:44589: Server oryx/127.19.194.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":50}
W20260806 18:34:50.175828 29711 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 18:34:50.171281 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.194.3:36369 (local address 127.19.194.62:44589)
0806 18:34:50.171389 (+ 108us) server_negotiation.cc:206] Beginning negotiation
0806 18:34:50.171393 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 18:34:50.171404 (+ 11us) server_negotiation.cc:407] Connection header received
0806 18:34:50.171492 (+ 88us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:34:50.171494 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:34:50.171523 (+ 29us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:34:50.171582 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:34:50.171883 (+ 301us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:34:50.172319 (+ 436us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:34:50.173002 (+ 683us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:34:50.173227 (+ 225us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:34:50.173296 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:34:50.175713 (+ 2417us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.19.194.3:36369: BlockingRecv error: recv got EOF from 127.19.194.3:36369 (error 108)
Metrics: {"server-negotiator.queue_time_us":41}
Aug 06 18:34:50 dist-test-slave-1xrh krb5kdc[28692](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.19.194.3@KRBTEST.COM for oryx/127.19.194.62@KRBTEST.COM, Server not found in Kerberos database
Aug 06 18:34:50 dist-test-slave-1xrh krb5kdc[28692](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.19.194.3@KRBTEST.COM for oryx/127.19.194.62@KRBTEST.COM, Server not found in Kerberos database
I20260806 18:34:50.180297 29712 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:34:50.176137 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.194.62:44589 (local address 127.19.194.3:52677)
0806 18:34:50.176255 (+ 118us) negotiation.cc:107] Waiting for socket to connect
0806 18:34:50.176268 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0806 18:34:50.176344 (+ 76us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:34:50.176552 (+ 208us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:34:50.176554 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:34:50.176671 (+ 117us) client_negotiation.cc:189] Negotiated authn=SASL
0806 18:34:50.176732 (+ 61us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:34:50.176736 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:34:50.177364 (+ 628us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:34:50.177366 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:34:50.177846 (+ 480us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:34:50.177852 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:34:50.178220 (+ 368us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:34:50.178222 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:34:50.178273 (+ 51us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:34:50.178626 (+ 353us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 18:34:50.178636 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0806 18:34:50.180222 (+ 1586us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.19.194.62:44589: Server oryx/127.19.194.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":48}
W20260806 18:34:50.180430 29711 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 18:34:50.176213 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.194.3:52677 (local address 127.19.194.62:44589)
0806 18:34:50.176330 (+ 117us) server_negotiation.cc:206] Beginning negotiation
0806 18:34:50.176333 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 18:34:50.176345 (+ 12us) server_negotiation.cc:407] Connection header received
0806 18:34:50.176401 (+ 56us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:34:50.176403 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:34:50.176432 (+ 29us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:34:50.176517 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:34:50.176827 (+ 310us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:34:50.177274 (+ 447us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:34:50.177939 (+ 665us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:34:50.178130 (+ 191us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:34:50.178200 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:34:50.180333 (+ 2133us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.19.194.3:52677: BlockingRecv error: recv got EOF from 127.19.194.3:52677 (error 108)
Metrics: {"server-negotiator.queue_time_us":61}
W20260806 18:34:50.180601 29708 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
Aug 06 18:34:50 dist-test-slave-1xrh krb5kdc[28692](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.19.194.1@KRBTEST.COM for oryx/127.19.194.62@KRBTEST.COM, Server not found in Kerberos database
Aug 06 18:34:50 dist-test-slave-1xrh krb5kdc[28692](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.19.194.1@KRBTEST.COM for oryx/127.19.194.62@KRBTEST.COM, Server not found in Kerberos database
I20260806 18:34:50.814517 29713 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:34:50.803222 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.194.62:44589 (local address 127.19.194.1:44331)
0806 18:34:50.803545 (+ 323us) negotiation.cc:107] Waiting for socket to connect
0806 18:34:50.803565 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0806 18:34:50.803682 (+ 117us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:34:50.804046 (+ 364us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:34:50.804071 (+ 25us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:34:50.804381 (+ 310us) client_negotiation.cc:189] Negotiated authn=SASL
0806 18:34:50.804524 (+ 143us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:34:50.804532 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:34:50.805423 (+ 891us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:34:50.805426 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:34:50.805990 (+ 564us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:34:50.805998 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:34:50.806420 (+ 422us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:34:50.806422 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:34:50.806483 (+ 61us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:34:50.812444 (+ 5961us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 18:34:50.812457 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0806 18:34:50.814390 (+ 1933us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.19.194.62:44589: Server oryx/127.19.194.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":207,"thread_start_us":98,"threads_started":1}
W20260806 18:34:50.814688 29714 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 18:34:50.803270 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.194.1:44331 (local address 127.19.194.62:44589)
0806 18:34:50.803591 (+ 321us) server_negotiation.cc:206] Beginning negotiation
0806 18:34:50.803600 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0806 18:34:50.803619 (+ 19us) server_negotiation.cc:407] Connection header received
0806 18:34:50.803826 (+ 207us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:34:50.803832 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:34:50.803895 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:34:50.803999 (+ 104us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:34:50.804671 (+ 672us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:34:50.805320 (+ 649us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:34:50.806104 (+ 784us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:34:50.806319 (+ 215us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:34:50.806393 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:34:50.814536 (+ 8143us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.19.194.1:44331: BlockingRecv error: recv got EOF from 127.19.194.1:44331 (error 108)
Metrics: {"server-negotiator.queue_time_us":207,"thread_start_us":88,"threads_started":1}
Aug 06 18:34:51 dist-test-slave-1xrh krb5kdc[28692](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.19.194.2@KRBTEST.COM for oryx/127.19.194.62@KRBTEST.COM, Server not found in Kerberos database
Aug 06 18:34:51 dist-test-slave-1xrh krb5kdc[28692](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.19.194.2@KRBTEST.COM for oryx/127.19.194.62@KRBTEST.COM, Server not found in Kerberos database
I20260806 18:34:51.010120 29715 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:34:51.004020 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.194.62:44589 (local address 127.19.194.2:54013)
0806 18:34:51.004299 (+ 279us) negotiation.cc:107] Waiting for socket to connect
0806 18:34:51.004320 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0806 18:34:51.004419 (+ 99us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:34:51.004689 (+ 270us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:34:51.004692 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:34:51.004912 (+ 220us) client_negotiation.cc:189] Negotiated authn=SASL
0806 18:34:51.005041 (+ 129us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:34:51.005048 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:34:51.005826 (+ 778us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:34:51.005831 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:34:51.006422 (+ 591us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:34:51.006431 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:34:51.006925 (+ 494us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:34:51.006929 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:34:51.007046 (+ 117us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:34:51.007674 (+ 628us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 18:34:51.007687 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0806 18:34:51.009947 (+ 2260us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.19.194.62:44589: Server oryx/127.19.194.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":196,"thread_start_us":82,"threads_started":1}
W20260806 18:34:51.010257 29714 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 18:34:51.004085 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.194.2:54013 (local address 127.19.194.62:44589)
0806 18:34:51.004229 (+ 144us) server_negotiation.cc:206] Beginning negotiation
0806 18:34:51.004233 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 18:34:51.004389 (+ 156us) server_negotiation.cc:407] Connection header received
0806 18:34:51.004488 (+ 99us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:34:51.004490 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:34:51.004544 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:34:51.004667 (+ 123us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:34:51.005172 (+ 505us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:34:51.005707 (+ 535us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:34:51.006567 (+ 860us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:34:51.006796 (+ 229us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:34:51.006907 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:34:51.010139 (+ 3232us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.19.194.2:54013: BlockingRecv error: recv got EOF from 127.19.194.2:54013 (error 108)
Metrics: {"server-negotiator.queue_time_us":53}
Aug 06 18:34:51 dist-test-slave-1xrh krb5kdc[28692](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.19.194.3@KRBTEST.COM for oryx/127.19.194.62@KRBTEST.COM, Server not found in Kerberos database
Aug 06 18:34:51 dist-test-slave-1xrh krb5kdc[28692](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.19.194.3@KRBTEST.COM for oryx/127.19.194.62@KRBTEST.COM, Server not found in Kerberos database
I20260806 18:34:51.186587 29716 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:34:51.181205 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.194.62:44589 (local address 127.19.194.3:45707)
0806 18:34:51.181467 (+ 262us) negotiation.cc:107] Waiting for socket to connect
0806 18:34:51.181503 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0806 18:34:51.181601 (+ 98us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:34:51.181850 (+ 249us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:34:51.181852 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:34:51.182028 (+ 176us) client_negotiation.cc:189] Negotiated authn=SASL
0806 18:34:51.182142 (+ 114us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:34:51.182148 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:34:51.182869 (+ 721us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:34:51.182871 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:34:51.183446 (+ 575us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:34:51.183454 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:34:51.183881 (+ 427us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:34:51.183884 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:34:51.183957 (+ 73us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:34:51.184532 (+ 575us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 18:34:51.184544 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0806 18:34:51.186471 (+ 1927us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.19.194.62:44589: Server oryx/127.19.194.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":179,"thread_start_us":94,"threads_started":1}
W20260806 18:34:51.186784 29714 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 18:34:51.181344 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.194.3:45707 (local address 127.19.194.62:44589)
0806 18:34:51.181495 (+ 151us) server_negotiation.cc:206] Beginning negotiation
0806 18:34:51.181498 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 18:34:51.181580 (+ 82us) server_negotiation.cc:407] Connection header received
0806 18:34:51.181692 (+ 112us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:34:51.181694 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:34:51.181740 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:34:51.181813 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:34:51.182250 (+ 437us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:34:51.182765 (+ 515us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:34:51.183542 (+ 777us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:34:51.183750 (+ 208us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:34:51.183859 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:34:51.186642 (+ 2783us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.19.194.3:45707: BlockingRecv error: recv got EOF from 127.19.194.3:45707 (error 108)
Metrics: {"server-negotiator.queue_time_us":56}
Aug 06 18:34:51 dist-test-slave-1xrh krb5kdc[28692](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.19.194.1@KRBTEST.COM for oryx/127.19.194.62@KRBTEST.COM, Server not found in Kerberos database
Aug 06 18:34:51 dist-test-slave-1xrh krb5kdc[28692](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.19.194.1@KRBTEST.COM for oryx/127.19.194.62@KRBTEST.COM, Server not found in Kerberos database
I20260806 18:34:51.820917 29717 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:34:51.815618 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.194.62:44589 (local address 127.19.194.1:40875)
0806 18:34:51.815905 (+ 287us) negotiation.cc:107] Waiting for socket to connect
0806 18:34:51.815919 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0806 18:34:51.816017 (+ 98us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:34:51.816278 (+ 261us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:34:51.816280 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:34:51.816478 (+ 198us) client_negotiation.cc:189] Negotiated authn=SASL
0806 18:34:51.816586 (+ 108us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:34:51.816592 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:34:51.817326 (+ 734us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:34:51.817328 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:34:51.817878 (+ 550us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:34:51.817886 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:34:51.818301 (+ 415us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:34:51.818303 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:34:51.818365 (+ 62us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:34:51.818960 (+ 595us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 18:34:51.818972 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0806 18:34:51.820779 (+ 1807us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.19.194.62:44589: Server oryx/127.19.194.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":195,"thread_start_us":96,"threads_started":1}
W20260806 18:34:51.821058 29718 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 18:34:51.815687 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.194.1:40875 (local address 127.19.194.62:44589)
0806 18:34:51.815934 (+ 247us) server_negotiation.cc:206] Beginning negotiation
0806 18:34:51.815938 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 18:34:51.815967 (+ 29us) server_negotiation.cc:407] Connection header received
0806 18:34:51.816107 (+ 140us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:34:51.816109 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:34:51.816155 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:34:51.816234 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:34:51.816700 (+ 466us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:34:51.817229 (+ 529us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:34:51.817982 (+ 753us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:34:51.818201 (+ 219us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:34:51.818276 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:34:51.820919 (+ 2643us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.19.194.1:40875: BlockingRecv error: recv got EOF from 127.19.194.1:40875 (error 108)
Metrics: {"server-negotiator.queue_time_us":181,"thread_start_us":94,"threads_started":1}
Aug 06 18:34:52 dist-test-slave-1xrh krb5kdc[28692](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.19.194.2@KRBTEST.COM for oryx/127.19.194.62@KRBTEST.COM, Server not found in Kerberos database
Aug 06 18:34:52 dist-test-slave-1xrh krb5kdc[28692](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.19.194.2@KRBTEST.COM for oryx/127.19.194.62@KRBTEST.COM, Server not found in Kerberos database
I20260806 18:34:52.016216 29719 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:34:52.010941 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.194.62:44589 (local address 127.19.194.2:38913)
0806 18:34:52.011261 (+ 320us) negotiation.cc:107] Waiting for socket to connect
0806 18:34:52.011274 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0806 18:34:52.011374 (+ 100us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:34:52.011611 (+ 237us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:34:52.011613 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:34:52.011793 (+ 180us) client_negotiation.cc:189] Negotiated authn=SASL
0806 18:34:52.011893 (+ 100us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:34:52.011899 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:34:52.012633 (+ 734us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:34:52.012636 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:34:52.013188 (+ 552us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:34:52.013196 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:34:52.013609 (+ 413us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:34:52.013611 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:34:52.013672 (+ 61us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:34:52.014181 (+ 509us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 18:34:52.014192 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0806 18:34:52.016101 (+ 1909us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.19.194.62:44589: Server oryx/127.19.194.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":240,"thread_start_us":119,"threads_started":1}
W20260806 18:34:52.016356 29718 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 18:34:52.011054 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.194.2:38913 (local address 127.19.194.62:44589)
0806 18:34:52.011222 (+ 168us) server_negotiation.cc:206] Beginning negotiation
0806 18:34:52.011225 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 18:34:52.011344 (+ 119us) server_negotiation.cc:407] Connection header received
0806 18:34:52.011454 (+ 110us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:34:52.011456 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:34:52.011507 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:34:52.011565 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:34:52.012013 (+ 448us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:34:52.012537 (+ 524us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:34:52.013291 (+ 754us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:34:52.013509 (+ 218us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:34:52.013584 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:34:52.016233 (+ 2649us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.19.194.2:38913: BlockingRecv error: recv got EOF from 127.19.194.2:38913 (error 108)
Metrics: {"server-negotiator.queue_time_us":56}
Aug 06 18:34:52 dist-test-slave-1xrh krb5kdc[28692](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.19.194.3@KRBTEST.COM for oryx/127.19.194.62@KRBTEST.COM, Server not found in Kerberos database
Aug 06 18:34:52 dist-test-slave-1xrh krb5kdc[28692](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.19.194.3@KRBTEST.COM for oryx/127.19.194.62@KRBTEST.COM, Server not found in Kerberos database
I20260806 18:34:52.193451 29720 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:34:52.187451 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.194.62:44589 (local address 127.19.194.3:38835)
0806 18:34:52.187800 (+ 349us) negotiation.cc:107] Waiting for socket to connect
0806 18:34:52.187816 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0806 18:34:52.187959 (+ 143us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:34:52.188248 (+ 289us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:34:52.188251 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:34:52.188483 (+ 232us) client_negotiation.cc:189] Negotiated authn=SASL
0806 18:34:52.188585 (+ 102us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:34:52.188591 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:34:52.189400 (+ 809us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:34:52.189405 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:34:52.190031 (+ 626us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:34:52.190040 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:34:52.190640 (+ 600us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:34:52.190644 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:34:52.190757 (+ 113us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:34:52.191376 (+ 619us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 18:34:52.191387 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0806 18:34:52.193334 (+ 1947us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.19.194.62:44589: Server oryx/127.19.194.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":271,"thread_start_us":85,"threads_started":1}
W20260806 18:34:52.193631 29718 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 18:34:52.187620 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.194.3:38835 (local address 127.19.194.62:44589)
0806 18:34:52.187776 (+ 156us) server_negotiation.cc:206] Beginning negotiation
0806 18:34:52.187781 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 18:34:52.187903 (+ 122us) server_negotiation.cc:407] Connection header received
0806 18:34:52.188083 (+ 180us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:34:52.188090 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:34:52.188137 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:34:52.188223 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:34:52.188713 (+ 490us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:34:52.189273 (+ 560us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:34:52.190171 (+ 898us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:34:52.190514 (+ 343us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:34:52.190633 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:34:52.193466 (+ 2833us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.19.194.3:38835: BlockingRecv error: recv got EOF from 127.19.194.3:38835 (error 108)
Metrics: {"server-negotiator.queue_time_us":63}
Aug 06 18:34:52 dist-test-slave-1xrh krb5kdc[28692](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.19.194.1@KRBTEST.COM for oryx/127.19.194.62@KRBTEST.COM, Server not found in Kerberos database
Aug 06 18:34:52 dist-test-slave-1xrh krb5kdc[28692](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.19.194.1@KRBTEST.COM for oryx/127.19.194.62@KRBTEST.COM, Server not found in Kerberos database
I20260806 18:34:52.827234 29721 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:34:52.821785 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.194.62:44589 (local address 127.19.194.1:46635)
0806 18:34:52.822121 (+ 336us) negotiation.cc:107] Waiting for socket to connect
0806 18:34:52.822159 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0806 18:34:52.822267 (+ 108us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:34:52.822534 (+ 267us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:34:52.822536 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:34:52.822721 (+ 185us) client_negotiation.cc:189] Negotiated authn=SASL
0806 18:34:52.822818 (+ 97us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:34:52.822823 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:34:52.823625 (+ 802us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:34:52.823628 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:34:52.824202 (+ 574us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:34:52.824210 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:34:52.824622 (+ 412us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:34:52.824624 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:34:52.824685 (+ 61us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:34:52.825178 (+ 493us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 18:34:52.825189 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0806 18:34:52.827103 (+ 1914us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.19.194.62:44589: Server oryx/127.19.194.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":243,"thread_start_us":120,"threads_started":1}
W20260806 18:34:52.827433 29722 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 18:34:52.821871 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.194.1:46635 (local address 127.19.194.62:44589)
0806 18:34:52.822246 (+ 375us) server_negotiation.cc:206] Beginning negotiation
0806 18:34:52.822249 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 18:34:52.822262 (+ 13us) server_negotiation.cc:407] Connection header received
0806 18:34:52.822338 (+ 76us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:34:52.822340 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:34:52.822388 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:34:52.822483 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:34:52.822897 (+ 414us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:34:52.823517 (+ 620us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:34:52.824303 (+ 786us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:34:52.824514 (+ 211us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:34:52.824595 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:34:52.827288 (+ 2693us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.19.194.1:46635: BlockingRecv error: recv got EOF from 127.19.194.1:46635 (error 108)
Metrics: {"server-negotiator.queue_time_us":307,"thread_start_us":152,"threads_started":1}
Aug 06 18:34:53 dist-test-slave-1xrh krb5kdc[28692](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.19.194.2@KRBTEST.COM for oryx/127.19.194.62@KRBTEST.COM, Server not found in Kerberos database
Aug 06 18:34:53 dist-test-slave-1xrh krb5kdc[28692](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.19.194.2@KRBTEST.COM for oryx/127.19.194.62@KRBTEST.COM, Server not found in Kerberos database
I20260806 18:34:53.022425 29723 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:34:53.017080 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.194.62:44589 (local address 127.19.194.2:51961)
0806 18:34:53.017352 (+ 272us) negotiation.cc:107] Waiting for socket to connect
0806 18:34:53.017365 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0806 18:34:53.017467 (+ 102us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:34:53.017691 (+ 224us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:34:53.017693 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:34:53.017884 (+ 191us) client_negotiation.cc:189] Negotiated authn=SASL
0806 18:34:53.017992 (+ 108us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:34:53.017997 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:34:53.018703 (+ 706us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:34:53.018705 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:34:53.019246 (+ 541us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:34:53.019255 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:34:53.019662 (+ 407us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:34:53.019665 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:34:53.019722 (+ 57us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:34:53.020184 (+ 462us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 18:34:53.020195 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0806 18:34:53.022264 (+ 2069us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.19.194.62:44589: Server oryx/127.19.194.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":179,"thread_start_us":86,"threads_started":1}
W20260806 18:34:53.022619 29722 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 18:34:53.017151 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.194.2:51961 (local address 127.19.194.62:44589)
0806 18:34:53.017290 (+ 139us) server_negotiation.cc:206] Beginning negotiation
0806 18:34:53.017294 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 18:34:53.017430 (+ 136us) server_negotiation.cc:407] Connection header received
0806 18:34:53.017538 (+ 108us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:34:53.017540 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:34:53.017584 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:34:53.017651 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:34:53.018092 (+ 441us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:34:53.018600 (+ 508us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:34:53.019351 (+ 751us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:34:53.019566 (+ 215us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:34:53.019640 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:34:53.022428 (+ 2788us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.19.194.2:51961: BlockingRecv error: recv got EOF from 127.19.194.2:51961 (error 108)
Metrics: {"server-negotiator.queue_time_us":52}
Aug 06 18:34:53 dist-test-slave-1xrh krb5kdc[28692](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.19.194.3@KRBTEST.COM for oryx/127.19.194.62@KRBTEST.COM, Server not found in Kerberos database
Aug 06 18:34:53 dist-test-slave-1xrh krb5kdc[28692](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.19.194.3@KRBTEST.COM for oryx/127.19.194.62@KRBTEST.COM, Server not found in Kerberos database
I20260806 18:34:53.199834 29724 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:34:53.194188 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.194.62:44589 (local address 127.19.194.3:53879)
0806 18:34:53.194472 (+ 284us) negotiation.cc:107] Waiting for socket to connect
0806 18:34:53.194485 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0806 18:34:53.194577 (+ 92us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:34:53.194822 (+ 245us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:34:53.194824 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:34:53.195014 (+ 190us) client_negotiation.cc:189] Negotiated authn=SASL
0806 18:34:53.195112 (+ 98us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:34:53.195117 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:34:53.195901 (+ 784us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:34:53.195903 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:34:53.196462 (+ 559us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:34:53.196469 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:34:53.196890 (+ 421us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:34:53.196892 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:34:53.196955 (+ 63us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:34:53.197441 (+ 486us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 18:34:53.197453 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0806 18:34:53.199653 (+ 2200us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.19.194.62:44589: Server oryx/127.19.194.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":204,"thread_start_us":117,"threads_started":1}
W20260806 18:34:53.199872 29722 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 18:34:53.194276 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.194.3:53879 (local address 127.19.194.62:44589)
0806 18:34:53.194420 (+ 144us) server_negotiation.cc:206] Beginning negotiation
0806 18:34:53.194424 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 18:34:53.194538 (+ 114us) server_negotiation.cc:407] Connection header received
0806 18:34:53.194653 (+ 115us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:34:53.194655 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:34:53.194701 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:34:53.194778 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:34:53.195243 (+ 465us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:34:53.195805 (+ 562us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:34:53.196565 (+ 760us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:34:53.196791 (+ 226us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:34:53.196866 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:34:53.199769 (+ 2903us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.19.194.3:53879: BlockingRecv error: recv got EOF from 127.19.194.3:53879 (error 108)
Metrics: {"server-negotiator.queue_time_us":56}
Aug 06 18:34:53 dist-test-slave-1xrh krb5kdc[28692](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.19.194.1@KRBTEST.COM for oryx/127.19.194.62@KRBTEST.COM, Server not found in Kerberos database
Aug 06 18:34:53 dist-test-slave-1xrh krb5kdc[28692](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.19.194.1@KRBTEST.COM for oryx/127.19.194.62@KRBTEST.COM, Server not found in Kerberos database
I20260806 18:34:53.833338 29725 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:34:53.828057 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.194.62:44589 (local address 127.19.194.1:46219)
0806 18:34:53.828374 (+ 317us) negotiation.cc:107] Waiting for socket to connect
0806 18:34:53.828390 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0806 18:34:53.828505 (+ 115us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:34:53.828769 (+ 264us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:34:53.828771 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:34:53.828947 (+ 176us) client_negotiation.cc:189] Negotiated authn=SASL
0806 18:34:53.829047 (+ 100us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:34:53.829053 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:34:53.829774 (+ 721us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:34:53.829776 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:34:53.830320 (+ 544us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:34:53.830327 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:34:53.830731 (+ 404us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:34:53.830734 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:34:53.830797 (+ 63us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:34:53.831436 (+ 639us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 18:34:53.831448 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0806 18:34:53.833227 (+ 1779us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.19.194.62:44589: Server oryx/127.19.194.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":212,"thread_start_us":88,"threads_started":1}
W20260806 18:34:53.833482 29726 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 18:34:53.828120 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.194.1:46219 (local address 127.19.194.62:44589)
0806 18:34:53.828395 (+ 275us) server_negotiation.cc:206] Beginning negotiation
0806 18:34:53.828399 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 18:34:53.828454 (+ 55us) server_negotiation.cc:407] Connection header received
0806 18:34:53.828606 (+ 152us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:34:53.828608 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:34:53.828652 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:34:53.828725 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:34:53.829164 (+ 439us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:34:53.829683 (+ 519us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:34:53.830421 (+ 738us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:34:53.830633 (+ 212us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:34:53.830706 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:34:53.833352 (+ 2646us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.19.194.1:46219: BlockingRecv error: recv got EOF from 127.19.194.1:46219 (error 108)
Metrics: {"server-negotiator.queue_time_us":197,"thread_start_us":66,"threads_started":1}
Aug 06 18:34:54 dist-test-slave-1xrh krb5kdc[28692](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.19.194.2@KRBTEST.COM for oryx/127.19.194.62@KRBTEST.COM, Server not found in Kerberos database
Aug 06 18:34:54 dist-test-slave-1xrh krb5kdc[28692](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.19.194.2@KRBTEST.COM for oryx/127.19.194.62@KRBTEST.COM, Server not found in Kerberos database
I20260806 18:34:54.035534 29727 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:34:54.023241 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.194.62:44589 (local address 127.19.194.2:58073)
0806 18:34:54.023584 (+ 343us) negotiation.cc:107] Waiting for socket to connect
0806 18:34:54.023596 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0806 18:34:54.023727 (+ 131us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:34:54.024035 (+ 308us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:34:54.024039 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:34:54.024238 (+ 199us) client_negotiation.cc:189] Negotiated authn=SASL
0806 18:34:54.024332 (+ 94us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:34:54.024337 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:34:54.025115 (+ 778us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:34:54.025120 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:34:54.025676 (+ 556us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:34:54.025684 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:34:54.026151 (+ 467us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:34:54.026155 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:34:54.026241 (+ 86us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:34:54.033142 (+ 6901us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 18:34:54.033154 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0806 18:34:54.035341 (+ 2187us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.19.194.62:44589: Server oryx/127.19.194.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":284,"thread_start_us":88,"threads_started":1}
W20260806 18:34:54.035573 29726 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 18:34:54.023307 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.194.2:58073 (local address 127.19.194.62:44589)
0806 18:34:54.023479 (+ 172us) server_negotiation.cc:206] Beginning negotiation
0806 18:34:54.023483 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 18:34:54.023674 (+ 191us) server_negotiation.cc:407] Connection header received
0806 18:34:54.023848 (+ 174us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:34:54.023877 (+ 29us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:34:54.023923 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:34:54.024010 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:34:54.024456 (+ 446us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:34:54.025002 (+ 546us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:34:54.025801 (+ 799us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:34:54.026035 (+ 234us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:34:54.026145 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:34:54.035471 (+ 9326us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.19.194.2:58073: BlockingRecv error: recv got EOF from 127.19.194.2:58073 (error 108)
Metrics: {"server-negotiator.queue_time_us":64}
Aug 06 18:34:54 dist-test-slave-1xrh krb5kdc[28692](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.19.194.3@KRBTEST.COM for oryx/127.19.194.62@KRBTEST.COM, Server not found in Kerberos database
Aug 06 18:34:54 dist-test-slave-1xrh krb5kdc[28692](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.19.194.3@KRBTEST.COM for oryx/127.19.194.62@KRBTEST.COM, Server not found in Kerberos database
I20260806 18:34:54.206624 29728 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:34:54.200690 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.194.62:44589 (local address 127.19.194.3:33287)
0806 18:34:54.200987 (+ 297us) negotiation.cc:107] Waiting for socket to connect
0806 18:34:54.201000 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0806 18:34:54.201098 (+ 98us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:34:54.201317 (+ 219us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:34:54.201319 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:34:54.201511 (+ 192us) client_negotiation.cc:189] Negotiated authn=SASL
0806 18:34:54.201610 (+ 99us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:34:54.201616 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:34:54.202364 (+ 748us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:34:54.202366 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:34:54.203000 (+ 634us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:34:54.203009 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:34:54.203591 (+ 582us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:34:54.203594 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:34:54.203660 (+ 66us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:34:54.204183 (+ 523us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 18:34:54.204195 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0806 18:34:54.206482 (+ 2287us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.19.194.62:44589: Server oryx/127.19.194.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":209,"thread_start_us":119,"threads_started":1}
W20260806 18:34:54.206768 29726 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 18:34:54.200801 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.194.3:33287 (local address 127.19.194.62:44589)
0806 18:34:54.200949 (+ 148us) server_negotiation.cc:206] Beginning negotiation
0806 18:34:54.200953 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 18:34:54.201052 (+ 99us) server_negotiation.cc:407] Connection header received
0806 18:34:54.201163 (+ 111us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:34:54.201165 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:34:54.201210 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:34:54.201280 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:34:54.201716 (+ 436us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:34:54.202269 (+ 553us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:34:54.203102 (+ 833us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:34:54.203461 (+ 359us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:34:54.203570 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:34:54.206617 (+ 3047us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.19.194.3:33287: BlockingRecv error: recv got EOF from 127.19.194.3:33287 (error 108)
Metrics: {"server-negotiator.queue_time_us":62}
Aug 06 18:34:54 dist-test-slave-1xrh krb5kdc[28692](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.19.194.1@KRBTEST.COM for oryx/127.19.194.62@KRBTEST.COM, Server not found in Kerberos database
Aug 06 18:34:54 dist-test-slave-1xrh krb5kdc[28692](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.19.194.1@KRBTEST.COM for oryx/127.19.194.62@KRBTEST.COM, Server not found in Kerberos database
I20260806 18:34:54.839478 29729 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:34:54.834200 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.194.62:44589 (local address 127.19.194.1:35035)
0806 18:34:54.834488 (+ 288us) negotiation.cc:107] Waiting for socket to connect
0806 18:34:54.834511 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0806 18:34:54.834596 (+ 85us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:34:54.834810 (+ 214us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:34:54.834812 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:34:54.834992 (+ 180us) client_negotiation.cc:189] Negotiated authn=SASL
0806 18:34:54.835097 (+ 105us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:34:54.835103 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:34:54.835885 (+ 782us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:34:54.835888 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:34:54.836550 (+ 662us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:34:54.836559 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:34:54.836984 (+ 425us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:34:54.836986 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:34:54.837054 (+ 68us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:34:54.837564 (+ 510us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 18:34:54.837576 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0806 18:34:54.839369 (+ 1793us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.19.194.62:44589: Server oryx/127.19.194.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":193,"thread_start_us":101,"threads_started":1}
W20260806 18:34:54.839615 29730 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 18:34:54.834242 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.194.1:35035 (local address 127.19.194.62:44589)
0806 18:34:54.834537 (+ 295us) server_negotiation.cc:206] Beginning negotiation
0806 18:34:54.834541 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 18:34:54.834556 (+ 15us) server_negotiation.cc:407] Connection header received
0806 18:34:54.834677 (+ 121us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:34:54.834679 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:34:54.834722 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:34:54.834798 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:34:54.835258 (+ 460us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:34:54.835788 (+ 530us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:34:54.836649 (+ 861us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:34:54.836887 (+ 238us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:34:54.836973 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:34:54.839488 (+ 2515us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.19.194.1:35035: BlockingRecv error: recv got EOF from 127.19.194.1:35035 (error 108)
Metrics: {"server-negotiator.queue_time_us":222,"thread_start_us":102,"threads_started":1}
Aug 06 18:34:55 dist-test-slave-1xrh krb5kdc[28692](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.19.194.2@KRBTEST.COM for oryx/127.19.194.62@KRBTEST.COM, Server not found in Kerberos database
Aug 06 18:34:55 dist-test-slave-1xrh krb5kdc[28692](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.19.194.2@KRBTEST.COM for oryx/127.19.194.62@KRBTEST.COM, Server not found in Kerberos database
I20260806 18:34:55.041755 29731 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:34:55.036251 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.194.62:44589 (local address 127.19.194.2:36307)
0806 18:34:55.036581 (+ 330us) negotiation.cc:107] Waiting for socket to connect
0806 18:34:55.036598 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0806 18:34:55.036723 (+ 125us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:34:55.036999 (+ 276us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:34:55.037001 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:34:55.037191 (+ 190us) client_negotiation.cc:189] Negotiated authn=SASL
0806 18:34:55.037291 (+ 100us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:34:55.037296 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:34:55.038163 (+ 867us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:34:55.038167 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:34:55.038909 (+ 742us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:34:55.038920 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:34:55.039344 (+ 424us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:34:55.039347 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:34:55.039405 (+ 58us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:34:55.039857 (+ 452us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 18:34:55.039867 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0806 18:34:55.041643 (+ 1776us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.19.194.62:44589: Server oryx/127.19.194.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":249,"thread_start_us":126,"threads_started":1}
W20260806 18:34:55.041927 29730 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 18:34:55.036410 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.194.2:36307 (local address 127.19.194.62:44589)
0806 18:34:55.036562 (+ 152us) server_negotiation.cc:206] Beginning negotiation
0806 18:34:55.036567 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 18:34:55.036689 (+ 122us) server_negotiation.cc:407] Connection header received
0806 18:34:55.036821 (+ 132us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:34:55.036823 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:34:55.036891 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:34:55.036967 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:34:55.037389 (+ 422us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:34:55.038006 (+ 617us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:34:55.039021 (+ 1015us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:34:55.039256 (+ 235us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:34:55.039328 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:34:55.041775 (+ 2447us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.19.194.2:36307: BlockingRecv error: recv got EOF from 127.19.194.2:36307 (error 108)
Metrics: {"server-negotiator.queue_time_us":54}
Aug 06 18:34:55 dist-test-slave-1xrh krb5kdc[28692](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.19.194.3@KRBTEST.COM for oryx/127.19.194.62@KRBTEST.COM, Server not found in Kerberos database
Aug 06 18:34:55 dist-test-slave-1xrh krb5kdc[28692](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.19.194.3@KRBTEST.COM for oryx/127.19.194.62@KRBTEST.COM, Server not found in Kerberos database
I20260806 18:34:55.213063 29732 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:34:55.207421 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.194.62:44589 (local address 127.19.194.3:44931)
0806 18:34:55.207739 (+ 318us) negotiation.cc:107] Waiting for socket to connect
0806 18:34:55.207761 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0806 18:34:55.207902 (+ 141us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:34:55.208190 (+ 288us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:34:55.208193 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:34:55.208421 (+ 228us) client_negotiation.cc:189] Negotiated authn=SASL
0806 18:34:55.208515 (+ 94us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:34:55.208520 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:34:55.209302 (+ 782us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:34:55.209307 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:34:55.209922 (+ 615us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:34:55.209930 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:34:55.210407 (+ 477us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:34:55.210411 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:34:55.210500 (+ 89us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:34:55.211025 (+ 525us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 18:34:55.211038 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0806 18:34:55.212938 (+ 1900us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.19.194.62:44589: Server oryx/127.19.194.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":243,"thread_start_us":89,"threads_started":1}
W20260806 18:34:55.213251 29730 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 18:34:55.207520 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.194.3:44931 (local address 127.19.194.62:44589)
0806 18:34:55.207670 (+ 150us) server_negotiation.cc:206] Beginning negotiation
0806 18:34:55.207675 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 18:34:55.207873 (+ 198us) server_negotiation.cc:407] Connection header received
0806 18:34:55.208010 (+ 137us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:34:55.208018 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:34:55.208075 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:34:55.208159 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:34:55.208651 (+ 492us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:34:55.209191 (+ 540us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:34:55.210056 (+ 865us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:34:55.210283 (+ 227us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:34:55.210389 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:34:55.213079 (+ 2690us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.19.194.3:44931: BlockingRecv error: recv got EOF from 127.19.194.3:44931 (error 108)
Metrics: {"server-negotiator.queue_time_us":52}
Aug 06 18:34:55 dist-test-slave-1xrh krb5kdc[28692](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.19.194.1@KRBTEST.COM for oryx/127.19.194.62@KRBTEST.COM, Server not found in Kerberos database
Aug 06 18:34:55 dist-test-slave-1xrh krb5kdc[28692](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.19.194.1@KRBTEST.COM for oryx/127.19.194.62@KRBTEST.COM, Server not found in Kerberos database
I20260806 18:34:55.845331 29733 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:34:55.840231 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.194.62:44589 (local address 127.19.194.1:53515)
0806 18:34:55.840501 (+ 270us) negotiation.cc:107] Waiting for socket to connect
0806 18:34:55.840513 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0806 18:34:55.840617 (+ 104us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:34:55.840837 (+ 220us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:34:55.840839 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:34:55.841016 (+ 177us) client_negotiation.cc:189] Negotiated authn=SASL
0806 18:34:55.841113 (+ 97us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:34:55.841118 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:34:55.841834 (+ 716us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:34:55.841836 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:34:55.842358 (+ 522us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:34:55.842364 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:34:55.842790 (+ 426us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:34:55.842792 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:34:55.842856 (+ 64us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:34:55.843452 (+ 596us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 18:34:55.843465 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0806 18:34:55.845221 (+ 1756us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.19.194.62:44589: Server oryx/127.19.194.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":166,"thread_start_us":84,"threads_started":1}
W20260806 18:34:55.845494 29734 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 18:34:55.840277 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.194.1:53515 (local address 127.19.194.62:44589)
0806 18:34:55.840519 (+ 242us) server_negotiation.cc:206] Beginning negotiation
0806 18:34:55.840522 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 18:34:55.840566 (+ 44us) server_negotiation.cc:407] Connection header received
0806 18:34:55.840690 (+ 124us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:34:55.840692 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:34:55.840734 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:34:55.840803 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:34:55.841201 (+ 398us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:34:55.841743 (+ 542us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:34:55.842447 (+ 704us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:34:55.842690 (+ 243us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:34:55.842762 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:34:55.845352 (+ 2590us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.19.194.1:53515: BlockingRecv error: recv got EOF from 127.19.194.1:53515 (error 108)
Metrics: {"server-negotiator.queue_time_us":172,"thread_start_us":86,"threads_started":1}
Aug 06 18:34:56 dist-test-slave-1xrh krb5kdc[28692](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.19.194.2@KRBTEST.COM for oryx/127.19.194.62@KRBTEST.COM, Server not found in Kerberos database
Aug 06 18:34:56 dist-test-slave-1xrh krb5kdc[28692](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.19.194.2@KRBTEST.COM for oryx/127.19.194.62@KRBTEST.COM, Server not found in Kerberos database
I20260806 18:34:56.052949 29735 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:34:56.042472 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.194.62:44589 (local address 127.19.194.2:49873)
0806 18:34:56.042771 (+ 299us) negotiation.cc:107] Waiting for socket to connect
0806 18:34:56.042788 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0806 18:34:56.042895 (+ 107us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:34:56.046834 (+ 3939us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:34:56.046837 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:34:56.047189 (+ 352us) client_negotiation.cc:189] Negotiated authn=SASL
0806 18:34:56.047348 (+ 159us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:34:56.047356 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:34:56.049312 (+ 1956us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:34:56.049318 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:34:56.049921 (+ 603us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:34:56.049930 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:34:56.050405 (+ 475us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:34:56.050409 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:34:56.050506 (+ 97us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:34:56.050986 (+ 480us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 18:34:56.050998 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0806 18:34:56.052834 (+ 1836us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.19.194.62:44589: Server oryx/127.19.194.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":204,"thread_start_us":95,"threads_started":1}
W20260806 18:34:56.053133 29734 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 18:34:56.045952 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.194.2:49873 (local address 127.19.194.62:44589)
0806 18:34:56.046603 (+ 651us) server_negotiation.cc:206] Beginning negotiation
0806 18:34:56.046610 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 18:34:56.046624 (+ 14us) server_negotiation.cc:407] Connection header received
0806 18:34:56.046665 (+ 41us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:34:56.046669 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:34:56.046717 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:34:56.046802 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:34:56.048657 (+ 1855us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:34:56.049196 (+ 539us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:34:56.050049 (+ 853us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:34:56.050290 (+ 241us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:34:56.050397 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:34:56.052977 (+ 2580us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.19.194.2:49873: BlockingRecv error: recv got EOF from 127.19.194.2:49873 (error 108)
Metrics: {"server-negotiator.queue_time_us":548}
Aug 06 18:34:56 dist-test-slave-1xrh krb5kdc[28692](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.19.194.3@KRBTEST.COM for oryx/127.19.194.62@KRBTEST.COM, Server not found in Kerberos database
Aug 06 18:34:56 dist-test-slave-1xrh krb5kdc[28692](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.19.194.3@KRBTEST.COM for oryx/127.19.194.62@KRBTEST.COM, Server not found in Kerberos database
I20260806 18:34:56.219668 29736 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:34:56.214012 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.194.62:44589 (local address 127.19.194.3:39215)
0806 18:34:56.214348 (+ 336us) negotiation.cc:107] Waiting for socket to connect
0806 18:34:56.214369 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0806 18:34:56.214503 (+ 134us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:34:56.214788 (+ 285us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:34:56.214791 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:34:56.215006 (+ 215us) client_negotiation.cc:189] Negotiated authn=SASL
0806 18:34:56.215106 (+ 100us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:34:56.215112 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:34:56.215888 (+ 776us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:34:56.215893 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:34:56.216438 (+ 545us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:34:56.216446 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:34:56.216904 (+ 458us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:34:56.216908 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:34:56.217015 (+ 107us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:34:56.217540 (+ 525us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 18:34:56.217552 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0806 18:34:56.219489 (+ 1937us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.19.194.62:44589: Server oryx/127.19.194.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":226,"thread_start_us":110,"threads_started":1}
W20260806 18:34:56.219704 29734 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 18:34:56.214118 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.194.3:39215 (local address 127.19.194.62:44589)
0806 18:34:56.214289 (+ 171us) server_negotiation.cc:206] Beginning negotiation
0806 18:34:56.214294 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 18:34:56.214444 (+ 150us) server_negotiation.cc:407] Connection header received
0806 18:34:56.214614 (+ 170us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:34:56.214618 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:34:56.214674 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:34:56.214761 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:34:56.215255 (+ 494us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:34:56.215773 (+ 518us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:34:56.216566 (+ 793us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:34:56.216789 (+ 223us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:34:56.216902 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:34:56.219607 (+ 2705us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.19.194.3:39215: BlockingRecv error: recv got EOF from 127.19.194.3:39215 (error 108)
Metrics: {"server-negotiator.queue_time_us":76}
Aug 06 18:34:56 dist-test-slave-1xrh krb5kdc[28692](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.19.194.1@KRBTEST.COM for oryx/127.19.194.62@KRBTEST.COM, Server not found in Kerberos database
Aug 06 18:34:56 dist-test-slave-1xrh krb5kdc[28692](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.19.194.1@KRBTEST.COM for oryx/127.19.194.62@KRBTEST.COM, Server not found in Kerberos database
I20260806 18:34:56.851559 29737 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:34:56.846174 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.194.62:44589 (local address 127.19.194.1:47981)
0806 18:34:56.846489 (+ 315us) negotiation.cc:107] Waiting for socket to connect
0806 18:34:56.846503 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0806 18:34:56.846606 (+ 103us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:34:56.846822 (+ 216us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:34:56.846825 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:34:56.847027 (+ 202us) client_negotiation.cc:189] Negotiated authn=SASL
0806 18:34:56.847126 (+ 99us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:34:56.847131 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:34:56.847898 (+ 767us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:34:56.847900 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:34:56.848438 (+ 538us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:34:56.848445 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:34:56.848917 (+ 472us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:34:56.848921 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:34:56.849006 (+ 85us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:34:56.849567 (+ 561us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 18:34:56.849579 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0806 18:34:56.851380 (+ 1801us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.19.194.62:44589: Server oryx/127.19.194.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":232,"thread_start_us":89,"threads_started":1}
W20260806 18:34:56.851624 29738 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 18:34:56.846340 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.194.1:47981 (local address 127.19.194.62:44589)
0806 18:34:56.846613 (+ 273us) server_negotiation.cc:206] Beginning negotiation
0806 18:34:56.846616 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 18:34:56.846630 (+ 14us) server_negotiation.cc:407] Connection header received
0806 18:34:56.846664 (+ 34us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:34:56.846666 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:34:56.846707 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:34:56.846784 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:34:56.847265 (+ 481us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:34:56.847802 (+ 537us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:34:56.848571 (+ 769us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:34:56.848804 (+ 233us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:34:56.848910 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:34:56.851504 (+ 2594us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.19.194.1:47981: BlockingRecv error: recv got EOF from 127.19.194.1:47981 (error 108)
Metrics: {"server-negotiator.queue_time_us":200,"thread_start_us":115,"threads_started":1}
Aug 06 18:34:57 dist-test-slave-1xrh krb5kdc[28692](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.19.194.2@KRBTEST.COM for oryx/127.19.194.62@KRBTEST.COM, Server not found in Kerberos database
Aug 06 18:34:57 dist-test-slave-1xrh krb5kdc[28692](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.19.194.2@KRBTEST.COM for oryx/127.19.194.62@KRBTEST.COM, Server not found in Kerberos database
I20260806 18:34:57.059438 29739 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:34:57.053685 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.194.62:44589 (local address 127.19.194.2:49927)
0806 18:34:57.054112 (+ 427us) negotiation.cc:107] Waiting for socket to connect
0806 18:34:57.054131 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0806 18:34:57.054254 (+ 123us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:34:57.054608 (+ 354us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:34:57.054612 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:34:57.054860 (+ 248us) client_negotiation.cc:189] Negotiated authn=SASL
0806 18:34:57.055014 (+ 154us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:34:57.055030 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:34:57.055853 (+ 823us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:34:57.055856 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:34:57.056435 (+ 579us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:34:57.056443 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:34:57.056839 (+ 396us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:34:57.056842 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:34:57.056904 (+ 62us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:34:57.057402 (+ 498us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 18:34:57.057414 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0806 18:34:57.059324 (+ 1910us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.19.194.62:44589: Server oryx/127.19.194.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":341,"thread_start_us":121,"threads_started":1}
W20260806 18:34:57.059573 29738 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 18:34:57.053880 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.194.2:49927 (local address 127.19.194.62:44589)
0806 18:34:57.054068 (+ 188us) server_negotiation.cc:206] Beginning negotiation
0806 18:34:57.054073 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 18:34:57.054212 (+ 139us) server_negotiation.cc:407] Connection header received
0806 18:34:57.054359 (+ 147us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:34:57.054363 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:34:57.054418 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:34:57.054530 (+ 112us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:34:57.055185 (+ 655us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:34:57.055753 (+ 568us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:34:57.056537 (+ 784us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:34:57.056744 (+ 207us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:34:57.056816 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:34:57.059455 (+ 2639us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.19.194.2:49927: BlockingRecv error: recv got EOF from 127.19.194.2:49927 (error 108)
Metrics: {"server-negotiator.queue_time_us":67}
Aug 06 18:34:57 dist-test-slave-1xrh krb5kdc[28692](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.19.194.3@KRBTEST.COM for oryx/127.19.194.62@KRBTEST.COM, Server not found in Kerberos database
Aug 06 18:34:57 dist-test-slave-1xrh krb5kdc[28692](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.19.194.3@KRBTEST.COM for oryx/127.19.194.62@KRBTEST.COM, Server not found in Kerberos database
I20260806 18:34:57.226333 29740 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:34:57.220469 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.194.62:44589 (local address 127.19.194.3:35001)
0806 18:34:57.220788 (+ 319us) negotiation.cc:107] Waiting for socket to connect
0806 18:34:57.220812 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0806 18:34:57.220914 (+ 102us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:34:57.221215 (+ 301us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:34:57.221218 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:34:57.221424 (+ 206us) client_negotiation.cc:189] Negotiated authn=SASL
0806 18:34:57.221537 (+ 113us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:34:57.221543 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:34:57.222452 (+ 909us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:34:57.222456 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:34:57.223094 (+ 638us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:34:57.223103 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:34:57.223596 (+ 493us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:34:57.223599 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:34:57.223664 (+ 65us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:34:57.224178 (+ 514us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 18:34:57.224190 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0806 18:34:57.226158 (+ 1968us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.19.194.62:44589: Server oryx/127.19.194.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":238,"thread_start_us":98,"threads_started":1}
W20260806 18:34:57.226375 29738 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 18:34:57.220560 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.194.3:35001 (local address 127.19.194.62:44589)
0806 18:34:57.220694 (+ 134us) server_negotiation.cc:206] Beginning negotiation
0806 18:34:57.220698 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 18:34:57.220878 (+ 180us) server_negotiation.cc:407] Connection header received
0806 18:34:57.220998 (+ 120us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:34:57.221001 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:34:57.221049 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:34:57.221118 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:34:57.221621 (+ 503us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:34:57.222333 (+ 712us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:34:57.223241 (+ 908us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:34:57.223493 (+ 252us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:34:57.223574 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:34:57.226277 (+ 2703us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.19.194.3:35001: BlockingRecv error: recv got EOF from 127.19.194.3:35001 (error 108)
Metrics: {"server-negotiator.queue_time_us":46}
Aug 06 18:34:57 dist-test-slave-1xrh krb5kdc[28692](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.19.194.1@KRBTEST.COM for oryx/127.19.194.62@KRBTEST.COM, Server not found in Kerberos database
Aug 06 18:34:57 dist-test-slave-1xrh krb5kdc[28692](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.19.194.1@KRBTEST.COM for oryx/127.19.194.62@KRBTEST.COM, Server not found in Kerberos database
I20260806 18:34:57.858177 29741 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:34:57.852522 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.194.62:44589 (local address 127.19.194.1:55639)
0806 18:34:57.852859 (+ 337us) negotiation.cc:107] Waiting for socket to connect
0806 18:34:57.852880 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0806 18:34:57.853001 (+ 121us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:34:57.853247 (+ 246us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:34:57.853249 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:34:57.853431 (+ 182us) client_negotiation.cc:189] Negotiated authn=SASL
0806 18:34:57.853529 (+ 98us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:34:57.853534 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:34:57.854290 (+ 756us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:34:57.854293 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:34:57.854841 (+ 548us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:34:57.854847 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:34:57.855298 (+ 451us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:34:57.855300 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:34:57.855365 (+ 65us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:34:57.855869 (+ 504us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 18:34:57.855880 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0806 18:34:57.858061 (+ 2181us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.19.194.62:44589: Server oryx/127.19.194.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":236,"thread_start_us":87,"threads_started":1}
W20260806 18:34:57.858361 29742 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 18:34:57.852767 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.194.1:55639 (local address 127.19.194.62:44589)
0806 18:34:57.853043 (+ 276us) server_negotiation.cc:206] Beginning negotiation
0806 18:34:57.853047 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 18:34:57.853059 (+ 12us) server_negotiation.cc:407] Connection header received
0806 18:34:57.853091 (+ 32us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:34:57.853093 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:34:57.853132 (+ 39us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:34:57.853204 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:34:57.853632 (+ 428us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:34:57.854192 (+ 560us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:34:57.854946 (+ 754us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:34:57.855192 (+ 246us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:34:57.855268 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:34:57.858199 (+ 2931us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.19.194.1:55639: BlockingRecv error: recv got EOF from 127.19.194.1:55639 (error 108)
Metrics: {"server-negotiator.queue_time_us":207,"thread_start_us":113,"threads_started":1}
Aug 06 18:34:58 dist-test-slave-1xrh krb5kdc[28692](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.19.194.2@KRBTEST.COM for oryx/127.19.194.62@KRBTEST.COM, Server not found in Kerberos database
Aug 06 18:34:58 dist-test-slave-1xrh krb5kdc[28692](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.19.194.2@KRBTEST.COM for oryx/127.19.194.62@KRBTEST.COM, Server not found in Kerberos database
I20260806 18:34:58.066407 29743 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:34:58.060444 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.194.62:44589 (local address 127.19.194.2:56493)
0806 18:34:58.060784 (+ 340us) negotiation.cc:107] Waiting for socket to connect
0806 18:34:58.060802 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0806 18:34:58.060922 (+ 120us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:34:58.061230 (+ 308us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:34:58.061233 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:34:58.061444 (+ 211us) client_negotiation.cc:189] Negotiated authn=SASL
0806 18:34:58.061548 (+ 104us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:34:58.061553 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:34:58.062486 (+ 933us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:34:58.062492 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:34:58.063025 (+ 533us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:34:58.063055 (+ 30us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:34:58.063688 (+ 633us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:34:58.063691 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:34:58.063797 (+ 106us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:34:58.064428 (+ 631us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 18:34:58.064440 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0806 18:34:58.066299 (+ 1859us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.19.194.62:44589: Server oryx/127.19.194.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":200,"thread_start_us":107,"threads_started":1}
W20260806 18:34:58.066588 29742 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 18:34:58.060641 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.194.2:56493 (local address 127.19.194.62:44589)
0806 18:34:58.060827 (+ 186us) server_negotiation.cc:206] Beginning negotiation
0806 18:34:58.060834 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 18:34:58.060854 (+ 20us) server_negotiation.cc:407] Connection header received
0806 18:34:58.061049 (+ 195us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:34:58.061053 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:34:58.061100 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:34:58.061197 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:34:58.061672 (+ 475us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:34:58.062364 (+ 692us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:34:58.063231 (+ 867us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:34:58.063569 (+ 338us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:34:58.063677 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:34:58.066459 (+ 2782us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.19.194.2:56493: BlockingRecv error: recv got EOF from 127.19.194.2:56493 (error 108)
Metrics: {"server-negotiator.queue_time_us":96}
Aug 06 18:34:58 dist-test-slave-1xrh krb5kdc[28692](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.19.194.3@KRBTEST.COM for oryx/127.19.194.62@KRBTEST.COM, Server not found in Kerberos database
Aug 06 18:34:58 dist-test-slave-1xrh krb5kdc[28692](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.19.194.3@KRBTEST.COM for oryx/127.19.194.62@KRBTEST.COM, Server not found in Kerberos database
I20260806 18:34:58.232478 29744 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:34:58.227279 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.194.62:44589 (local address 127.19.194.3:40551)
0806 18:34:58.227599 (+ 320us) negotiation.cc:107] Waiting for socket to connect
0806 18:34:58.227613 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0806 18:34:58.227734 (+ 121us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:34:58.227986 (+ 252us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:34:58.227989 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:34:58.228168 (+ 179us) client_negotiation.cc:189] Negotiated authn=SASL
0806 18:34:58.228267 (+ 99us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:34:58.228272 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:34:58.228978 (+ 706us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:34:58.228980 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:34:58.229539 (+ 559us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:34:58.229547 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:34:58.229942 (+ 395us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:34:58.229944 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:34:58.230007 (+ 63us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:34:58.230485 (+ 478us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 18:34:58.230497 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0806 18:34:58.232339 (+ 1842us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.19.194.62:44589: Server oryx/127.19.194.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":235,"thread_start_us":119,"threads_started":1}
W20260806 18:34:58.232661 29742 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 18:34:58.227321 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.194.3:40551 (local address 127.19.194.62:44589)
0806 18:34:58.227490 (+ 169us) server_negotiation.cc:206] Beginning negotiation
0806 18:34:58.227495 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 18:34:58.227707 (+ 212us) server_negotiation.cc:407] Connection header received
0806 18:34:58.227810 (+ 103us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:34:58.227815 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:34:58.227873 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:34:58.227956 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:34:58.228365 (+ 409us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:34:58.228887 (+ 522us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:34:58.229637 (+ 750us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:34:58.229845 (+ 208us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:34:58.229917 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:34:58.232505 (+ 2588us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.19.194.3:40551: BlockingRecv error: recv got EOF from 127.19.194.3:40551 (error 108)
Metrics: {"server-negotiator.queue_time_us":61}
Aug 06 18:34:58 dist-test-slave-1xrh krb5kdc[28692](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.19.194.1@KRBTEST.COM for oryx/127.19.194.62@KRBTEST.COM, Server not found in Kerberos database
Aug 06 18:34:58 dist-test-slave-1xrh krb5kdc[28692](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.19.194.1@KRBTEST.COM for oryx/127.19.194.62@KRBTEST.COM, Server not found in Kerberos database
I20260806 18:34:58.865065 29745 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:34:58.859006 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.194.62:44589 (local address 127.19.194.1:48867)
0806 18:34:58.859334 (+ 328us) negotiation.cc:107] Waiting for socket to connect
0806 18:34:58.859348 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0806 18:34:58.859441 (+ 93us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:34:58.859695 (+ 254us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:34:58.859697 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:34:58.859884 (+ 187us) client_negotiation.cc:189] Negotiated authn=SASL
0806 18:34:58.860001 (+ 117us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:34:58.860007 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:34:58.860780 (+ 773us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:34:58.860791 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:34:58.861416 (+ 625us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:34:58.861425 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:34:58.861901 (+ 476us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:34:58.861904 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:34:58.861977 (+ 73us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:34:58.862492 (+ 515us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 18:34:58.862505 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0806 18:34:58.864916 (+ 2411us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.19.194.62:44589: Server oryx/127.19.194.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":243,"thread_start_us":137,"threads_started":1}
W20260806 18:34:58.865154 29746 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 18:34:58.859097 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.194.1:48867 (local address 127.19.194.62:44589)
0806 18:34:58.859391 (+ 294us) server_negotiation.cc:206] Beginning negotiation
0806 18:34:58.859394 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 18:34:58.859408 (+ 14us) server_negotiation.cc:407] Connection header received
0806 18:34:58.859532 (+ 124us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:34:58.859534 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:34:58.859580 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:34:58.859658 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:34:58.860105 (+ 447us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:34:58.860652 (+ 547us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:34:58.861532 (+ 880us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:34:58.861756 (+ 224us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:34:58.861845 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:34:58.865051 (+ 3206us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.19.194.1:48867: BlockingRecv error: recv got EOF from 127.19.194.1:48867 (error 108)
Metrics: {"server-negotiator.queue_time_us":213,"thread_start_us":72,"threads_started":1}
Aug 06 18:34:59 dist-test-slave-1xrh krb5kdc[28692](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.19.194.2@KRBTEST.COM for oryx/127.19.194.62@KRBTEST.COM, Server not found in Kerberos database
Aug 06 18:34:59 dist-test-slave-1xrh krb5kdc[28692](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.19.194.2@KRBTEST.COM for oryx/127.19.194.62@KRBTEST.COM, Server not found in Kerberos database
I20260806 18:34:59.080448 29747 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:34:59.067323 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.194.62:44589 (local address 127.19.194.2:54997)
0806 18:34:59.067681 (+ 358us) negotiation.cc:107] Waiting for socket to connect
0806 18:34:59.067694 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0806 18:34:59.067835 (+ 141us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:34:59.068118 (+ 283us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:34:59.068121 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:34:59.068340 (+ 219us) client_negotiation.cc:189] Negotiated authn=SASL
0806 18:34:59.068446 (+ 106us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:34:59.068452 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:34:59.069370 (+ 918us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:34:59.069373 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:34:59.070122 (+ 749us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:34:59.070135 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:34:59.070608 (+ 473us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:34:59.070611 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:34:59.070672 (+ 61us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:34:59.078208 (+ 7536us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 18:34:59.078221 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0806 18:34:59.080273 (+ 2052us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.19.194.62:44589: Server oryx/127.19.194.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":281,"thread_start_us":81,"threads_started":1}
W20260806 18:34:59.080606 29746 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 18:34:59.067428 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.194.2:54997 (local address 127.19.194.62:44589)
0806 18:34:59.067582 (+ 154us) server_negotiation.cc:206] Beginning negotiation
0806 18:34:59.067586 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 18:34:59.067775 (+ 189us) server_negotiation.cc:407] Connection header received
0806 18:34:59.067940 (+ 165us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:34:59.067943 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:34:59.067993 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:34:59.068095 (+ 102us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:34:59.068574 (+ 479us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:34:59.069262 (+ 688us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:34:59.070253 (+ 991us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:34:59.070508 (+ 255us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:34:59.070592 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:34:59.080459 (+ 9867us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.19.194.2:54997: BlockingRecv error: recv got EOF from 127.19.194.2:54997 (error 108)
Metrics: {"server-negotiator.queue_time_us":56}
Aug 06 18:34:59 dist-test-slave-1xrh krb5kdc[28692](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.19.194.3@KRBTEST.COM for oryx/127.19.194.62@KRBTEST.COM, Server not found in Kerberos database
Aug 06 18:34:59 dist-test-slave-1xrh krb5kdc[28692](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.19.194.3@KRBTEST.COM for oryx/127.19.194.62@KRBTEST.COM, Server not found in Kerberos database
I20260806 18:34:59.239164 29748 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:34:59.233286 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.194.62:44589 (local address 127.19.194.3:50171)
0806 18:34:59.233562 (+ 276us) negotiation.cc:107] Waiting for socket to connect
0806 18:34:59.233591 (+ 29us) client_negotiation.cc:174] Beginning negotiation
0806 18:34:59.233687 (+ 96us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:34:59.233952 (+ 265us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:34:59.233954 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:34:59.234182 (+ 228us) client_negotiation.cc:189] Negotiated authn=SASL
0806 18:34:59.234298 (+ 116us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:34:59.234304 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:34:59.235199 (+ 895us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:34:59.235204 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:34:59.235869 (+ 665us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:34:59.235878 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:34:59.236370 (+ 492us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:34:59.236373 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:34:59.236436 (+ 63us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:34:59.237056 (+ 620us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 18:34:59.237067 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0806 18:34:59.239005 (+ 1938us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.19.194.62:44589: Server oryx/127.19.194.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":194,"thread_start_us":91,"threads_started":1}
W20260806 18:34:59.239312 29746 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 18:34:59.233337 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.194.3:50171 (local address 127.19.194.62:44589)
0806 18:34:59.233474 (+ 137us) server_negotiation.cc:206] Beginning negotiation
0806 18:34:59.233477 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 18:34:59.233665 (+ 188us) server_negotiation.cc:407] Connection header received
0806 18:34:59.233790 (+ 125us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:34:59.233792 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:34:59.233842 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:34:59.233909 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:34:59.234400 (+ 491us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:34:59.235008 (+ 608us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:34:59.235974 (+ 966us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:34:59.236230 (+ 256us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:34:59.236340 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:34:59.239201 (+ 2861us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.19.194.3:50171: BlockingRecv error: recv got EOF from 127.19.194.3:50171 (error 108)
Metrics: {"server-negotiator.queue_time_us":53}
Aug 06 18:34:59 dist-test-slave-1xrh krb5kdc[28692](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.19.194.1@KRBTEST.COM for oryx/127.19.194.62@KRBTEST.COM, Server not found in Kerberos database
Aug 06 18:34:59 dist-test-slave-1xrh krb5kdc[28692](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.19.194.1@KRBTEST.COM for oryx/127.19.194.62@KRBTEST.COM, Server not found in Kerberos database
I20260806 18:34:59.871999 29749 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:34:59.866114 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.194.62:44589 (local address 127.19.194.1:35811)
0806 18:34:59.866458 (+ 344us) negotiation.cc:107] Waiting for socket to connect
0806 18:34:59.866473 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0806 18:34:59.866558 (+ 85us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:34:59.866828 (+ 270us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:34:59.866830 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:34:59.867041 (+ 211us) client_negotiation.cc:189] Negotiated authn=SASL
0806 18:34:59.867230 (+ 189us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:34:59.867238 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:34:59.867963 (+ 725us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:34:59.867965 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:34:59.868527 (+ 562us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:34:59.868535 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:34:59.868944 (+ 409us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:34:59.868946 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:34:59.869007 (+ 61us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:34:59.869504 (+ 497us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 18:34:59.869516 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0806 18:34:59.871805 (+ 2289us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.19.194.62:44589: Server oryx/127.19.194.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":262,"thread_start_us":101,"threads_started":1}
W20260806 18:34:59.872045 29750 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 18:34:59.866243 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.194.1:35811 (local address 127.19.194.62:44589)
0806 18:34:59.866513 (+ 270us) server_negotiation.cc:206] Beginning negotiation
0806 18:34:59.866516 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 18:34:59.866530 (+ 14us) server_negotiation.cc:407] Connection header received
0806 18:34:59.866649 (+ 119us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:34:59.866651 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:34:59.866693 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:34:59.866767 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:34:59.867334 (+ 567us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:34:59.867868 (+ 534us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:34:59.868626 (+ 758us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:34:59.868839 (+ 213us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:34:59.868921 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:34:59.871926 (+ 3005us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.19.194.1:35811: BlockingRecv error: recv got EOF from 127.19.194.1:35811 (error 108)
Metrics: {"server-negotiator.queue_time_us":189,"thread_start_us":74,"threads_started":1}
Aug 06 18:35:00 dist-test-slave-1xrh krb5kdc[28692](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.19.194.2@KRBTEST.COM for oryx/127.19.194.62@KRBTEST.COM, Server not found in Kerberos database
Aug 06 18:35:00 dist-test-slave-1xrh krb5kdc[28692](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.19.194.2@KRBTEST.COM for oryx/127.19.194.62@KRBTEST.COM, Server not found in Kerberos database
I20260806 18:35:00.087219 29751 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:35:00.081412 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.194.62:44589 (local address 127.19.194.2:43787)
0806 18:35:00.081715 (+ 303us) negotiation.cc:107] Waiting for socket to connect
0806 18:35:00.081730 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0806 18:35:00.081863 (+ 133us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:35:00.082136 (+ 273us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:35:00.082139 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:35:00.082354 (+ 215us) client_negotiation.cc:189] Negotiated authn=SASL
0806 18:35:00.082459 (+ 105us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:35:00.082465 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:35:00.083255 (+ 790us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:35:00.083261 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:35:00.083845 (+ 584us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:35:00.083853 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:35:00.084318 (+ 465us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:35:00.084322 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:35:00.084407 (+ 85us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:35:00.084958 (+ 551us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 18:35:00.084987 (+ 29us) client_negotiation.cc:656] calling sasl_client_start()
0806 18:35:00.087019 (+ 2032us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.19.194.62:44589: Server oryx/127.19.194.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":225,"thread_start_us":108,"threads_started":1}
W20260806 18:35:00.087348 29750 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 18:35:00.081473 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.194.2:43787 (local address 127.19.194.62:44589)
0806 18:35:00.081645 (+ 172us) server_negotiation.cc:206] Beginning negotiation
0806 18:35:00.081651 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 18:35:00.081811 (+ 160us) server_negotiation.cc:407] Connection header received
0806 18:35:00.081962 (+ 151us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:35:00.081966 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:35:00.082024 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:35:00.082110 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:35:00.082579 (+ 469us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:35:00.083083 (+ 504us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:35:00.083971 (+ 888us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:35:00.084202 (+ 231us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:35:00.084312 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:35:00.087235 (+ 2923us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.19.194.2:43787: BlockingRecv error: recv got EOF from 127.19.194.2:43787 (error 108)
Metrics: {"server-negotiator.queue_time_us":57}
Aug 06 18:35:00 dist-test-slave-1xrh krb5kdc[28692](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.19.194.3@KRBTEST.COM for oryx/127.19.194.62@KRBTEST.COM, Server not found in Kerberos database
Aug 06 18:35:00 dist-test-slave-1xrh krb5kdc[28692](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.19.194.3@KRBTEST.COM for oryx/127.19.194.62@KRBTEST.COM, Server not found in Kerberos database
I20260806 18:35:00.245604 29752 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:35:00.240041 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.194.62:44589 (local address 127.19.194.3:52655)
0806 18:35:00.240355 (+ 314us) negotiation.cc:107] Waiting for socket to connect
0806 18:35:00.240372 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0806 18:35:00.240507 (+ 135us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:35:00.240808 (+ 301us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:35:00.240812 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:35:00.241029 (+ 217us) client_negotiation.cc:189] Negotiated authn=SASL
0806 18:35:00.241129 (+ 100us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:35:00.241135 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:35:00.241888 (+ 753us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:35:00.241892 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:35:00.242455 (+ 563us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:35:00.242463 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:35:00.242934 (+ 471us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:35:00.242938 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:35:00.243021 (+ 83us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:35:00.243584 (+ 563us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 18:35:00.243596 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0806 18:35:00.245422 (+ 1826us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.19.194.62:44589: Server oryx/127.19.194.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":230,"thread_start_us":105,"threads_started":1}
W20260806 18:35:00.245643 29750 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 18:35:00.240205 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.194.3:52655 (local address 127.19.194.62:44589)
0806 18:35:00.240355 (+ 150us) server_negotiation.cc:206] Beginning negotiation
0806 18:35:00.240361 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 18:35:00.240452 (+ 91us) server_negotiation.cc:407] Connection header received
0806 18:35:00.240642 (+ 190us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:35:00.240645 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:35:00.240693 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:35:00.240776 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:35:00.241244 (+ 468us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:35:00.241774 (+ 530us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:35:00.242595 (+ 821us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:35:00.242818 (+ 223us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:35:00.242927 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:35:00.245541 (+ 2614us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.19.194.3:52655: BlockingRecv error: recv got EOF from 127.19.194.3:52655 (error 108)
Metrics: {"server-negotiator.queue_time_us":57}
Aug 06 18:35:00 dist-test-slave-1xrh krb5kdc[28692](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.19.194.1@KRBTEST.COM for oryx/127.19.194.62@KRBTEST.COM, Server not found in Kerberos database
Aug 06 18:35:00 dist-test-slave-1xrh krb5kdc[28692](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.19.194.1@KRBTEST.COM for oryx/127.19.194.62@KRBTEST.COM, Server not found in Kerberos database
I20260806 18:35:00.878077 29753 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:35:00.872830 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.194.62:44589 (local address 127.19.194.1:43805)
0806 18:35:00.873117 (+ 287us) negotiation.cc:107] Waiting for socket to connect
0806 18:35:00.873131 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0806 18:35:00.873232 (+ 101us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:35:00.873438 (+ 206us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:35:00.873440 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:35:00.873631 (+ 191us) client_negotiation.cc:189] Negotiated authn=SASL
0806 18:35:00.873743 (+ 112us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:35:00.873749 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:35:00.874441 (+ 692us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:35:00.874443 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:35:00.874976 (+ 533us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:35:00.874983 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:35:00.875397 (+ 414us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:35:00.875400 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:35:00.875471 (+ 71us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:35:00.875960 (+ 489us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 18:35:00.875971 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0806 18:35:00.877893 (+ 1922us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.19.194.62:44589: Server oryx/127.19.194.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":205,"thread_start_us":88,"threads_started":1}
W20260806 18:35:00.878170 29754 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 18:35:00.872882 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.194.1:43805 (local address 127.19.194.62:44589)
0806 18:35:00.873133 (+ 251us) server_negotiation.cc:206] Beginning negotiation
0806 18:35:00.873136 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 18:35:00.873200 (+ 64us) server_negotiation.cc:407] Connection header received
0806 18:35:00.873308 (+ 108us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:35:00.873310 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:35:00.873355 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:35:00.873427 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:35:00.873842 (+ 415us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:35:00.874352 (+ 510us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:35:00.875073 (+ 721us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:35:00.875296 (+ 223us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:35:00.875368 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:35:00.878033 (+ 2665us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.19.194.1:43805: BlockingRecv error: recv got EOF from 127.19.194.1:43805 (error 108)
Metrics: {"server-negotiator.queue_time_us":172,"thread_start_us":66,"threads_started":1}
Aug 06 18:35:01 dist-test-slave-1xrh krb5kdc[28692](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.19.194.2@KRBTEST.COM for oryx/127.19.194.62@KRBTEST.COM, Server not found in Kerberos database
Aug 06 18:35:01 dist-test-slave-1xrh krb5kdc[28692](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.19.194.2@KRBTEST.COM for oryx/127.19.194.62@KRBTEST.COM, Server not found in Kerberos database
I20260806 18:35:01.093461 29755 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:35:01.087990 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.194.62:44589 (local address 127.19.194.2:50123)
0806 18:35:01.088286 (+ 296us) negotiation.cc:107] Waiting for socket to connect
0806 18:35:01.088299 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0806 18:35:01.088427 (+ 128us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:35:01.088720 (+ 293us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:35:01.088723 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:35:01.088932 (+ 209us) client_negotiation.cc:189] Negotiated authn=SASL
0806 18:35:01.089031 (+ 99us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:35:01.089037 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:35:01.089794 (+ 757us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:35:01.089799 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:35:01.090366 (+ 567us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:35:01.090373 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:35:01.090862 (+ 489us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:35:01.090866 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:35:01.090954 (+ 88us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:35:01.091519 (+ 565us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 18:35:01.091533 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0806 18:35:01.093291 (+ 1758us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.19.194.62:44589: Server oryx/127.19.194.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":207,"thread_start_us":90,"threads_started":1}
W20260806 18:35:01.093554 29754 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 18:35:01.088057 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.194.2:50123 (local address 127.19.194.62:44589)
0806 18:35:01.088231 (+ 174us) server_negotiation.cc:206] Beginning negotiation
0806 18:35:01.088236 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 18:35:01.088374 (+ 138us) server_negotiation.cc:407] Connection header received
0806 18:35:01.088558 (+ 184us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:35:01.088561 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:35:01.088614 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:35:01.088696 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:35:01.089154 (+ 458us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:35:01.089681 (+ 527us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:35:01.090484 (+ 803us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:35:01.090741 (+ 257us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:35:01.090845 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:35:01.093422 (+ 2577us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.19.194.2:50123: BlockingRecv error: recv got EOF from 127.19.194.2:50123 (error 108)
Metrics: {"server-negotiator.queue_time_us":71}
Aug 06 18:35:01 dist-test-slave-1xrh krb5kdc[28692](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.19.194.3@KRBTEST.COM for oryx/127.19.194.62@KRBTEST.COM, Server not found in Kerberos database
Aug 06 18:35:01 dist-test-slave-1xrh krb5kdc[28692](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.19.194.3@KRBTEST.COM for oryx/127.19.194.62@KRBTEST.COM, Server not found in Kerberos database
I20260806 18:35:01.251909 29756 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:35:01.246487 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.194.62:44589 (local address 127.19.194.3:49271)
0806 18:35:01.246768 (+ 281us) negotiation.cc:107] Waiting for socket to connect
0806 18:35:01.246782 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0806 18:35:01.246931 (+ 149us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:35:01.247218 (+ 287us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:35:01.247220 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:35:01.247411 (+ 191us) client_negotiation.cc:189] Negotiated authn=SASL
0806 18:35:01.247513 (+ 102us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:35:01.247519 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:35:01.248225 (+ 706us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:35:01.248227 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:35:01.248773 (+ 546us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:35:01.248780 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:35:01.249179 (+ 399us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:35:01.249181 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:35:01.249245 (+ 64us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:35:01.249831 (+ 586us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 18:35:01.249843 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0806 18:35:01.251779 (+ 1936us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.19.194.62:44589: Server oryx/127.19.194.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":191,"thread_start_us":90,"threads_started":1}
W20260806 18:35:01.252069 29754 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 18:35:01.246552 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.194.3:49271 (local address 127.19.194.62:44589)
0806 18:35:01.246741 (+ 189us) server_negotiation.cc:206] Beginning negotiation
0806 18:35:01.246747 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 18:35:01.246921 (+ 174us) server_negotiation.cc:407] Connection header received
0806 18:35:01.247043 (+ 122us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:35:01.247045 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:35:01.247094 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:35:01.247168 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:35:01.247618 (+ 450us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:35:01.248130 (+ 512us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:35:01.248870 (+ 740us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:35:01.249081 (+ 211us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:35:01.249154 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:35:01.251915 (+ 2761us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.19.194.3:49271: BlockingRecv error: recv got EOF from 127.19.194.3:49271 (error 108)
Metrics: {"server-negotiator.queue_time_us":63}
Aug 06 18:35:01 dist-test-slave-1xrh krb5kdc[28692](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.19.194.1@KRBTEST.COM for oryx/127.19.194.62@KRBTEST.COM, Server not found in Kerberos database
Aug 06 18:35:01 dist-test-slave-1xrh krb5kdc[28692](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.19.194.1@KRBTEST.COM for oryx/127.19.194.62@KRBTEST.COM, Server not found in Kerberos database
I20260806 18:35:01.884289 29757 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:35:01.879070 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.194.62:44589 (local address 127.19.194.1:52709)
0806 18:35:01.879382 (+ 312us) negotiation.cc:107] Waiting for socket to connect
0806 18:35:01.879396 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0806 18:35:01.879510 (+ 114us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:35:01.879755 (+ 245us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:35:01.879756 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:35:01.879942 (+ 186us) client_negotiation.cc:189] Negotiated authn=SASL
0806 18:35:01.880082 (+ 140us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:35:01.880093 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:35:01.880850 (+ 757us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:35:01.880852 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:35:01.881407 (+ 555us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:35:01.881415 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:35:01.881827 (+ 412us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:35:01.881829 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:35:01.881891 (+ 62us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:35:01.882388 (+ 497us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 18:35:01.882400 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0806 18:35:01.884174 (+ 1774us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.19.194.62:44589: Server oryx/127.19.194.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":228,"thread_start_us":117,"threads_started":1}
W20260806 18:35:01.884438 29758 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 18:35:01.879154 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.194.1:52709 (local address 127.19.194.62:44589)
0806 18:35:01.879400 (+ 246us) server_negotiation.cc:206] Beginning negotiation
0806 18:35:01.879404 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 18:35:01.879477 (+ 73us) server_negotiation.cc:407] Connection header received
0806 18:35:01.879590 (+ 113us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:35:01.879592 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:35:01.879638 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:35:01.879716 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:35:01.880224 (+ 508us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:35:01.880756 (+ 532us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:35:01.881511 (+ 755us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:35:01.881731 (+ 220us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:35:01.881805 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:35:01.884306 (+ 2501us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.19.194.1:52709: BlockingRecv error: recv got EOF from 127.19.194.1:52709 (error 108)
Metrics: {"server-negotiator.queue_time_us":167,"thread_start_us":66,"threads_started":1}
Aug 06 18:35:02 dist-test-slave-1xrh krb5kdc[28692](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.19.194.2@KRBTEST.COM for oryx/127.19.194.62@KRBTEST.COM, Server not found in Kerberos database
Aug 06 18:35:02 dist-test-slave-1xrh krb5kdc[28692](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.19.194.2@KRBTEST.COM for oryx/127.19.194.62@KRBTEST.COM, Server not found in Kerberos database
I20260806 18:35:02.099921 29759 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:35:02.094258 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.194.62:44589 (local address 127.19.194.2:51123)
0806 18:35:02.094622 (+ 364us) negotiation.cc:107] Waiting for socket to connect
0806 18:35:02.094644 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0806 18:35:02.094797 (+ 153us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:35:02.095081 (+ 284us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:35:02.095084 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:35:02.095342 (+ 258us) client_negotiation.cc:189] Negotiated authn=SASL
0806 18:35:02.095440 (+ 98us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:35:02.095446 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:35:02.096222 (+ 776us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:35:02.096227 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:35:02.096801 (+ 574us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:35:02.096809 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:35:02.097280 (+ 471us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:35:02.097284 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:35:02.097374 (+ 90us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:35:02.097878 (+ 504us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 18:35:02.097890 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0806 18:35:02.099751 (+ 1861us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.19.194.62:44589: Server oryx/127.19.194.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":282,"thread_start_us":84,"threads_started":1}
W20260806 18:35:02.099977 29758 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 18:35:02.094375 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.194.2:51123 (local address 127.19.194.62:44589)
0806 18:35:02.094576 (+ 201us) server_negotiation.cc:206] Beginning negotiation
0806 18:35:02.094582 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 18:35:02.094732 (+ 150us) server_negotiation.cc:407] Connection header received
0806 18:35:02.094902 (+ 170us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:35:02.094906 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:35:02.094961 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:35:02.095063 (+ 102us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:35:02.095567 (+ 504us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:35:02.096107 (+ 540us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:35:02.096926 (+ 819us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:35:02.097156 (+ 230us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:35:02.097262 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:35:02.099871 (+ 2609us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.19.194.2:51123: BlockingRecv error: recv got EOF from 127.19.194.2:51123 (error 108)
Metrics: {"server-negotiator.queue_time_us":92}
Aug 06 18:35:02 dist-test-slave-1xrh krb5kdc[28692](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.19.194.3@KRBTEST.COM for oryx/127.19.194.62@KRBTEST.COM, Server not found in Kerberos database
Aug 06 18:35:02 dist-test-slave-1xrh krb5kdc[28692](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.19.194.3@KRBTEST.COM for oryx/127.19.194.62@KRBTEST.COM, Server not found in Kerberos database
I20260806 18:35:02.258556 29760 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:35:02.252785 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.194.62:44589 (local address 127.19.194.3:51165)
0806 18:35:02.253110 (+ 325us) negotiation.cc:107] Waiting for socket to connect
0806 18:35:02.253124 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0806 18:35:02.253264 (+ 140us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:35:02.253598 (+ 334us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:35:02.253601 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:35:02.253855 (+ 254us) client_negotiation.cc:189] Negotiated authn=SASL
0806 18:35:02.253962 (+ 107us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:35:02.253971 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:35:02.254959 (+ 988us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:35:02.254964 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:35:02.255548 (+ 584us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:35:02.255556 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:35:02.256008 (+ 452us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:35:02.256011 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:35:02.256071 (+ 60us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:35:02.256631 (+ 560us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 18:35:02.256643 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0806 18:35:02.258432 (+ 1789us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.19.194.62:44589: Server oryx/127.19.194.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":232,"thread_start_us":98,"threads_started":1}
W20260806 18:35:02.258749 29758 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 18:35:02.252857 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.194.3:51165 (local address 127.19.194.62:44589)
0806 18:35:02.253038 (+ 181us) server_negotiation.cc:206] Beginning negotiation
0806 18:35:02.253043 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 18:35:02.253205 (+ 162us) server_negotiation.cc:407] Connection header received
0806 18:35:02.253400 (+ 195us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:35:02.253405 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:35:02.253469 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:35:02.253571 (+ 102us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:35:02.254078 (+ 507us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:35:02.254842 (+ 764us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:35:02.255653 (+ 811us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:35:02.255903 (+ 250us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:35:02.255993 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:35:02.258575 (+ 2582us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.19.194.3:51165: BlockingRecv error: recv got EOF from 127.19.194.3:51165 (error 108)
Metrics: {"server-negotiator.queue_time_us":72}
Aug 06 18:35:02 dist-test-slave-1xrh krb5kdc[28692](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.19.194.1@KRBTEST.COM for oryx/127.19.194.62@KRBTEST.COM, Server not found in Kerberos database
Aug 06 18:35:02 dist-test-slave-1xrh krb5kdc[28692](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.19.194.1@KRBTEST.COM for oryx/127.19.194.62@KRBTEST.COM, Server not found in Kerberos database
I20260806 18:35:02.890697 29761 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:35:02.885086 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.194.62:44589 (local address 127.19.194.1:51395)
0806 18:35:02.885411 (+ 325us) negotiation.cc:107] Waiting for socket to connect
0806 18:35:02.885428 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0806 18:35:02.885561 (+ 133us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:35:02.885851 (+ 290us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:35:02.885854 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:35:02.886094 (+ 240us) client_negotiation.cc:189] Negotiated authn=SASL
0806 18:35:02.886192 (+ 98us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:35:02.886198 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:35:02.886939 (+ 741us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:35:02.886944 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:35:02.887570 (+ 626us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:35:02.887580 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:35:02.888058 (+ 478us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:35:02.888062 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:35:02.888154 (+ 92us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:35:02.888680 (+ 526us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 18:35:02.888692 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0806 18:35:02.890500 (+ 1808us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.19.194.62:44589: Server oryx/127.19.194.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":232,"thread_start_us":109,"threads_started":1}
W20260806 18:35:02.890777 29762 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 18:35:02.885147 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.194.1:51395 (local address 127.19.194.62:44589)
0806 18:35:02.885411 (+ 264us) server_negotiation.cc:206] Beginning negotiation
0806 18:35:02.885417 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 18:35:02.885485 (+ 68us) server_negotiation.cc:407] Connection header received
0806 18:35:02.885666 (+ 181us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:35:02.885682 (+ 16us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:35:02.885734 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:35:02.885825 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:35:02.886315 (+ 490us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:35:02.886830 (+ 515us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:35:02.887699 (+ 869us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:35:02.887937 (+ 238us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:35:02.888058 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:35:02.890635 (+ 2577us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.19.194.1:51395: BlockingRecv error: recv got EOF from 127.19.194.1:51395 (error 108)
Metrics: {"server-negotiator.queue_time_us":176,"thread_start_us":64,"threads_started":1}
Aug 06 18:35:03 dist-test-slave-1xrh krb5kdc[28692](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.19.194.2@KRBTEST.COM for oryx/127.19.194.62@KRBTEST.COM, Server not found in Kerberos database
Aug 06 18:35:03 dist-test-slave-1xrh krb5kdc[28692](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.19.194.2@KRBTEST.COM for oryx/127.19.194.62@KRBTEST.COM, Server not found in Kerberos database
I20260806 18:35:03.106501 29763 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:35:03.100782 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.194.62:44589 (local address 127.19.194.2:54097)
0806 18:35:03.101115 (+ 333us) negotiation.cc:107] Waiting for socket to connect
0806 18:35:03.101132 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0806 18:35:03.101287 (+ 155us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:35:03.101579 (+ 292us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:35:03.101582 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:35:03.101814 (+ 232us) client_negotiation.cc:189] Negotiated authn=SASL
0806 18:35:03.101907 (+ 93us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:35:03.101913 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:35:03.102658 (+ 745us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:35:03.102664 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:35:03.103285 (+ 621us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:35:03.103311 (+ 26us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:35:03.103810 (+ 499us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:35:03.103814 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:35:03.103922 (+ 108us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:35:03.104442 (+ 520us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 18:35:03.104454 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0806 18:35:03.106327 (+ 1873us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.19.194.62:44589: Server oryx/127.19.194.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":250,"thread_start_us":104,"threads_started":1}
W20260806 18:35:03.106570 29762 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 18:35:03.100948 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.194.2:54097 (local address 127.19.194.62:44589)
0806 18:35:03.101116 (+ 168us) server_negotiation.cc:206] Beginning negotiation
0806 18:35:03.101122 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 18:35:03.101220 (+ 98us) server_negotiation.cc:407] Connection header received
0806 18:35:03.101391 (+ 171us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:35:03.101394 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:35:03.101446 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:35:03.101551 (+ 105us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:35:03.102029 (+ 478us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:35:03.102546 (+ 517us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:35:03.103404 (+ 858us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:35:03.103687 (+ 283us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:35:03.103805 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:35:03.106452 (+ 2647us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.19.194.2:54097: BlockingRecv error: recv got EOF from 127.19.194.2:54097 (error 108)
Metrics: {"server-negotiator.queue_time_us":64}
Aug 06 18:35:03 dist-test-slave-1xrh krb5kdc[28692](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.19.194.3@KRBTEST.COM for oryx/127.19.194.62@KRBTEST.COM, Server not found in Kerberos database
Aug 06 18:35:03 dist-test-slave-1xrh krb5kdc[28692](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.19.194.3@KRBTEST.COM for oryx/127.19.194.62@KRBTEST.COM, Server not found in Kerberos database
I20260806 18:35:03.264375 29764 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:35:03.259295 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.194.62:44589 (local address 127.19.194.3:47813)
0806 18:35:03.259567 (+ 272us) negotiation.cc:107] Waiting for socket to connect
0806 18:35:03.259581 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0806 18:35:03.259680 (+ 99us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:35:03.259907 (+ 227us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:35:03.259909 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:35:03.260099 (+ 190us) client_negotiation.cc:189] Negotiated authn=SASL
0806 18:35:03.260193 (+ 94us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:35:03.260198 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:35:03.260873 (+ 675us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:35:03.260875 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:35:03.261424 (+ 549us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:35:03.261431 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:35:03.261825 (+ 394us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:35:03.261827 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:35:03.261887 (+ 60us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:35:03.262374 (+ 487us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 18:35:03.262385 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0806 18:35:03.264257 (+ 1872us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.19.194.62:44589: Server oryx/127.19.194.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":195,"thread_start_us":68,"threads_started":1}
W20260806 18:35:03.264518 29762 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 18:35:03.259389 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.194.3:47813 (local address 127.19.194.62:44589)
0806 18:35:03.259569 (+ 180us) server_negotiation.cc:206] Beginning negotiation
0806 18:35:03.259572 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 18:35:03.259649 (+ 77us) server_negotiation.cc:407] Connection header received
0806 18:35:03.259757 (+ 108us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:35:03.259760 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:35:03.259805 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:35:03.259869 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:35:03.260288 (+ 419us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:35:03.260786 (+ 498us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:35:03.261521 (+ 735us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:35:03.261730 (+ 209us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:35:03.261803 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:35:03.264395 (+ 2592us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.19.194.3:47813: BlockingRecv error: recv got EOF from 127.19.194.3:47813 (error 108)
Metrics: {"server-negotiator.queue_time_us":72}
Aug 06 18:35:03 dist-test-slave-1xrh krb5kdc[28692](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.19.194.1@KRBTEST.COM for oryx/127.19.194.62@KRBTEST.COM, Server not found in Kerberos database
Aug 06 18:35:03 dist-test-slave-1xrh krb5kdc[28692](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.19.194.1@KRBTEST.COM for oryx/127.19.194.62@KRBTEST.COM, Server not found in Kerberos database
I20260806 18:35:03.896863 29765 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:35:03.891587 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.194.62:44589 (local address 127.19.194.1:49915)
0806 18:35:03.891896 (+ 309us) negotiation.cc:107] Waiting for socket to connect
0806 18:35:03.891909 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0806 18:35:03.892019 (+ 110us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:35:03.892259 (+ 240us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:35:03.892261 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:35:03.892453 (+ 192us) client_negotiation.cc:189] Negotiated authn=SASL
0806 18:35:03.892562 (+ 109us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:35:03.892567 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:35:03.893289 (+ 722us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:35:03.893292 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:35:03.893859 (+ 567us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:35:03.893866 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:35:03.894265 (+ 399us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:35:03.894267 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:35:03.894328 (+ 61us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:35:03.894815 (+ 487us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 18:35:03.894826 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0806 18:35:03.896738 (+ 1912us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.19.194.62:44589: Server oryx/127.19.194.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":229,"thread_start_us":95,"threads_started":1}
W20260806 18:35:03.897018 29766 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 18:35:03.891713 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.194.1:49915 (local address 127.19.194.62:44589)
0806 18:35:03.892026 (+ 313us) server_negotiation.cc:206] Beginning negotiation
0806 18:35:03.892030 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 18:35:03.892043 (+ 13us) server_negotiation.cc:407] Connection header received
0806 18:35:03.892078 (+ 35us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:35:03.892080 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:35:03.892121 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:35:03.892193 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:35:03.892667 (+ 474us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:35:03.893200 (+ 533us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:35:03.893953 (+ 753us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:35:03.894174 (+ 221us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:35:03.894244 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:35:03.896875 (+ 2631us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.19.194.1:49915: BlockingRecv error: recv got EOF from 127.19.194.1:49915 (error 108)
Metrics: {"server-negotiator.queue_time_us":240,"thread_start_us":82,"threads_started":1}
Aug 06 18:35:04 dist-test-slave-1xrh krb5kdc[28692](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.19.194.2@KRBTEST.COM for oryx/127.19.194.62@KRBTEST.COM, Server not found in Kerberos database
Aug 06 18:35:04 dist-test-slave-1xrh krb5kdc[28692](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.19.194.2@KRBTEST.COM for oryx/127.19.194.62@KRBTEST.COM, Server not found in Kerberos database
I20260806 18:35:04.119292 29767 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:35:04.107312 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.194.62:44589 (local address 127.19.194.2:41231)
0806 18:35:04.107619 (+ 307us) negotiation.cc:107] Waiting for socket to connect
0806 18:35:04.107632 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0806 18:35:04.107734 (+ 102us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:35:04.107973 (+ 239us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:35:04.107975 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:35:04.108164 (+ 189us) client_negotiation.cc:189] Negotiated authn=SASL
0806 18:35:04.108264 (+ 100us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:35:04.108270 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:35:04.108978 (+ 708us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:35:04.108980 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:35:04.109578 (+ 598us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:35:04.109586 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:35:04.109985 (+ 399us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:35:04.109987 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:35:04.110050 (+ 63us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:35:04.116794 (+ 6744us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 18:35:04.116806 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0806 18:35:04.119093 (+ 2287us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.19.194.62:44589: Server oryx/127.19.194.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":229,"thread_start_us":128,"threads_started":1}
W20260806 18:35:04.119356 29766 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 18:35:04.107393 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.194.2:41231 (local address 127.19.194.62:44589)
0806 18:35:04.107562 (+ 169us) server_negotiation.cc:206] Beginning negotiation
0806 18:35:04.107568 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 18:35:04.107706 (+ 138us) server_negotiation.cc:407] Connection header received
0806 18:35:04.107814 (+ 108us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:35:04.107816 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:35:04.107863 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:35:04.107930 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:35:04.108363 (+ 433us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:35:04.108885 (+ 522us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:35:04.109680 (+ 795us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:35:04.109889 (+ 209us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:35:04.109970 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:35:04.119247 (+ 9277us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.19.194.2:41231: BlockingRecv error: recv got EOF from 127.19.194.2:41231 (error 108)
Metrics: {"server-negotiator.queue_time_us":69}
Aug 06 18:35:04 dist-test-slave-1xrh krb5kdc[28692](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.19.194.3@KRBTEST.COM for oryx/127.19.194.62@KRBTEST.COM, Server not found in Kerberos database
Aug 06 18:35:04 dist-test-slave-1xrh krb5kdc[28692](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.19.194.3@KRBTEST.COM for oryx/127.19.194.62@KRBTEST.COM, Server not found in Kerberos database
I20260806 18:35:04.271000 29768 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:35:04.265192 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.194.62:44589 (local address 127.19.194.3:52865)
0806 18:35:04.265474 (+ 282us) negotiation.cc:107] Waiting for socket to connect
0806 18:35:04.265486 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0806 18:35:04.265577 (+ 91us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:35:04.265816 (+ 239us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:35:04.265818 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:35:04.266033 (+ 215us) client_negotiation.cc:189] Negotiated authn=SASL
0806 18:35:04.266131 (+ 98us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:35:04.266142 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:35:04.267088 (+ 946us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:35:04.267092 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:35:04.267966 (+ 874us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:35:04.267979 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:35:04.268410 (+ 431us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:35:04.268413 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:35:04.268482 (+ 69us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:35:04.268982 (+ 500us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 18:35:04.268994 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0806 18:35:04.270880 (+ 1886us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.19.194.62:44589: Server oryx/127.19.194.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":194,"thread_start_us":107,"threads_started":1}
W20260806 18:35:04.271193 29766 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 18:35:04.265398 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.194.3:52865 (local address 127.19.194.62:44589)
0806 18:35:04.265543 (+ 145us) server_negotiation.cc:206] Beginning negotiation
0806 18:35:04.265547 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 18:35:04.265561 (+ 14us) server_negotiation.cc:407] Connection header received
0806 18:35:04.265659 (+ 98us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:35:04.265661 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:35:04.265708 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:35:04.265776 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:35:04.266256 (+ 480us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:35:04.266930 (+ 674us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:35:04.268085 (+ 1155us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:35:04.268302 (+ 217us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:35:04.268383 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:35:04.271017 (+ 2634us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.19.194.3:52865: BlockingRecv error: recv got EOF from 127.19.194.3:52865 (error 108)
Metrics: {"server-negotiator.queue_time_us":54}
Aug 06 18:35:04 dist-test-slave-1xrh krb5kdc[28692](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.19.194.1@KRBTEST.COM for oryx/127.19.194.62@KRBTEST.COM, Server not found in Kerberos database
Aug 06 18:35:04 dist-test-slave-1xrh krb5kdc[28692](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.19.194.1@KRBTEST.COM for oryx/127.19.194.62@KRBTEST.COM, Server not found in Kerberos database
I20260806 18:35:04.903769 29769 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:35:04.897870 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.194.62:44589 (local address 127.19.194.1:60815)
0806 18:35:04.898167 (+ 297us) negotiation.cc:107] Waiting for socket to connect
0806 18:35:04.898180 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0806 18:35:04.898265 (+ 85us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:35:04.898515 (+ 250us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:35:04.898517 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:35:04.898765 (+ 248us) client_negotiation.cc:189] Negotiated authn=SASL
0806 18:35:04.898867 (+ 102us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:35:04.898873 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:35:04.899745 (+ 872us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:35:04.899748 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:35:04.900423 (+ 675us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:35:04.900433 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:35:04.900971 (+ 538us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:35:04.900975 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:35:04.901124 (+ 149us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:35:04.901627 (+ 503us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 18:35:04.901639 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0806 18:35:04.903597 (+ 1958us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.19.194.62:44589: Server oryx/127.19.194.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":211,"thread_start_us":89,"threads_started":1}
W20260806 18:35:04.903914 29771 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 18:35:04.897941 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.194.1:60815 (local address 127.19.194.62:44589)
0806 18:35:04.898209 (+ 268us) server_negotiation.cc:206] Beginning negotiation
0806 18:35:04.898213 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 18:35:04.898227 (+ 14us) server_negotiation.cc:407] Connection header received
0806 18:35:04.898352 (+ 125us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:35:04.898354 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:35:04.898399 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:35:04.898482 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:35:04.898974 (+ 492us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:35:04.899591 (+ 617us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:35:04.900538 (+ 947us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:35:04.900846 (+ 308us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:35:04.900953 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:35:04.903727 (+ 2774us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.19.194.1:60815: BlockingRecv error: recv got EOF from 127.19.194.1:60815 (error 108)
Metrics: {"server-negotiator.queue_time_us":204,"thread_start_us":65,"threads_started":1}
Aug 06 18:35:05 dist-test-slave-1xrh krb5kdc[28692](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.19.194.2@KRBTEST.COM for oryx/127.19.194.62@KRBTEST.COM, Server not found in Kerberos database
Aug 06 18:35:05 dist-test-slave-1xrh krb5kdc[28692](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.19.194.2@KRBTEST.COM for oryx/127.19.194.62@KRBTEST.COM, Server not found in Kerberos database
I20260806 18:35:05.125887 29772 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:35:05.120192 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.194.62:44589 (local address 127.19.194.2:46239)
0806 18:35:05.120542 (+ 350us) negotiation.cc:107] Waiting for socket to connect
0806 18:35:05.120554 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0806 18:35:05.120684 (+ 130us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:35:05.120952 (+ 268us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:35:05.120955 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:35:05.121165 (+ 210us) client_negotiation.cc:189] Negotiated authn=SASL
0806 18:35:05.121326 (+ 161us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:35:05.121333 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:35:05.122088 (+ 755us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:35:05.122092 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:35:05.122749 (+ 657us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:35:05.122758 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:35:05.123281 (+ 523us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:35:05.123285 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:35:05.123383 (+ 98us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:35:05.123922 (+ 539us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 18:35:05.123934 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0806 18:35:05.125715 (+ 1781us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.19.194.62:44589: Server oryx/127.19.194.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":289,"thread_start_us":92,"threads_started":1}
W20260806 18:35:05.125943 29771 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 18:35:05.120251 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.194.2:46239 (local address 127.19.194.62:44589)
0806 18:35:05.120419 (+ 168us) server_negotiation.cc:206] Beginning negotiation
0806 18:35:05.120424 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 18:35:05.120626 (+ 202us) server_negotiation.cc:407] Connection header received
0806 18:35:05.120781 (+ 155us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:35:05.120784 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:35:05.120839 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:35:05.120926 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:35:05.121439 (+ 513us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:35:05.121972 (+ 533us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:35:05.122873 (+ 901us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:35:05.123107 (+ 234us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:35:05.123277 (+ 170us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:35:05.125835 (+ 2558us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.19.194.2:46239: BlockingRecv error: recv got EOF from 127.19.194.2:46239 (error 108)
Metrics: {"server-negotiator.queue_time_us":66}
I20260806 18:35:05.164573 20232 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJW69tn/build/debug/bin/kudu with pid 29284
I20260806 18:35:05.171928 20232 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJW69tn/build/debug/bin/kudu with pid 29440
I20260806 18:35:05.178773 20232 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJW69tn/build/debug/bin/kudu with pid 29576
I20260806 18:35:05.184669 20232 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJW69tn/build/debug/bin/kudu with pid 29213
2026-08-06T18:35:05Z chronyd exiting
[ OK ] SecurityITest.TestMismatchingPrincipals (18007 ms)
[ RUN ] SecurityITest.TestRequireAuthenticationInsecureCluster
2026-08-06T18:35:05Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T18:35:05Z Disabled control of system clock
I20260806 18:35:05.215571 20232 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJW69tn/build/debug/bin/kudu
/tmp/dist-test-taskJW69tn/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786041246094603-20232-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786041246094603-20232-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786041246094603-20232-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786041246094603-20232-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.19.194.62:34005
--webserver_interface=127.19.194.62
--webserver_port=0
--builtin_ntp_servers=127.19.194.20:38573
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.19.194.62:34005
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {}
W20260806 18:35:05.326174 29776 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 18:35:05.326437 29776 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 18:35:05.326473 29776 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 18:35:05.330643 29776 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 18:35:05.330710 29776 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 18:35:05.330725 29776 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 18:35:05.330765 29776 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 18:35:05.330794 29776 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 18:35:05.335347 29776 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.194.20:38573
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786041246094603-20232-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786041246094603-20232-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.19.194.62:34005
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.19.194.62:34005
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786041246094603-20232-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.19.194.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29776
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786041246094603-20232-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 17:59:13 UTC on ccycloud.quasar-wvsnzx.root.comops.site
build id 4571028
I20260806 18:35:05.336369 29776 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 18:35:05.337699 29776 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 18:35:05.346287 29783 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 18:35:05.346285 29776 server_base.cc:1034] running on GCE node
W20260806 18:35:05.346318 29785 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:35:05.346282 29782 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 18:35:05.347074 29776 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 18:35:05.348124 29776 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 18:35:05.349323 29776 hybrid_clock.cc:648] HybridClock initialized: now 1786041305349307 us; error 27 us; skew 500 ppm
I20260806 18:35:05.351073 29776 webserver.cc:460] Webserver started at http://127.19.194.62:44499/ using document root <none> and password file <none>
I20260806 18:35:05.351653 29776 fs_manager.cc:359] Metadata directory not provided
I20260806 18:35:05.351713 29776 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 18:35:05.351871 29776 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 18:35:05.353730 29776 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786041246094603-20232-0/minicluster-data/master-0/data/instance:
uuid: "9673a668f18b4e298809f6afecc58f4a"
format_stamp: "Formatted at 2026-08-06 18:35:05 on dist-test-slave-1xrh"
server_key: "13aed620d0fa1341b9edfb199806405c"
server_key_iv: "2142b60920f3a6b7a8e4f69fa9f1d028"
server_key_version: "encryptionkey@0"
I20260806 18:35:05.354210 29776 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786041246094603-20232-0/minicluster-data/master-0/wal/instance:
uuid: "9673a668f18b4e298809f6afecc58f4a"
format_stamp: "Formatted at 2026-08-06 18:35:05 on dist-test-slave-1xrh"
server_key: "13aed620d0fa1341b9edfb199806405c"
server_key_iv: "2142b60920f3a6b7a8e4f69fa9f1d028"
server_key_version: "encryptionkey@0"
I20260806 18:35:05.357666 29776 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.002s sys 0.004s
I20260806 18:35:05.360095 29791 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 18:35:05.361258 29776 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.000s sys 0.003s
I20260806 18:35:05.361399 29776 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786041246094603-20232-0/minicluster-data/master-0/data,/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786041246094603-20232-0/minicluster-data/master-0/wal
uuid: "9673a668f18b4e298809f6afecc58f4a"
format_stamp: "Formatted at 2026-08-06 18:35:05 on dist-test-slave-1xrh"
server_key: "13aed620d0fa1341b9edfb199806405c"
server_key_iv: "2142b60920f3a6b7a8e4f69fa9f1d028"
server_key_version: "encryptionkey@0"
I20260806 18:35:05.361537 29776 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786041246094603-20232-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786041246094603-20232-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786041246094603-20232-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 18:35:05.391266 29776 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 18:35:05.392026 29776 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 18:35:05.392232 29776 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 18:35:05.399006 29776 rpc_server.cc:307] RPC server started. Bound to: 127.19.194.62:34005
I20260806 18:35:05.399044 29843 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.194.62:34005 every 8 connection(s)
I20260806 18:35:05.399959 29776 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786041246094603-20232-0/minicluster-data/master-0/data/info.pb
I20260806 18:35:05.400987 20232 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJW69tn/build/debug/bin/kudu as pid 29776
I20260806 18:35:05.401119 20232 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786041246094603-20232-0/minicluster-data/master-0/wal/instance
I20260806 18:35:05.401428 20232 external_mini_cluster.cc:1428] Setting key 3984fc0afad0396b93c7d133b22c6a76
I20260806 18:35:05.403460 29844 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 18:35:05.409971 29844 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9673a668f18b4e298809f6afecc58f4a: Bootstrap starting.
I20260806 18:35:05.411808 29847 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:35:05.402884 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59394 (local address 127.19.194.62:34005)
0806 18:35:05.403573 (+ 689us) server_negotiation.cc:206] Beginning negotiation
0806 18:35:05.403592 (+ 19us) server_negotiation.cc:399] Waiting for connection header
0806 18:35:05.403633 (+ 41us) server_negotiation.cc:407] Connection header received
0806 18:35:05.404511 (+ 878us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:35:05.404532 (+ 21us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:35:05.405025 (+ 493us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:35:05.405358 (+ 333us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:35:05.405753 (+ 395us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:35:05.407195 (+ 1442us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:35:05.408139 (+ 944us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:35:05.408822 (+ 683us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:35:05.409003 (+ 181us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:35:05.409801 (+ 798us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:35:05.409807 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:35:05.409821 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 18:35:05.410293 (+ 472us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:35:05.410357 (+ 64us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 18:35:05.410381 (+ 24us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:35:05.410469 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0806 18:35:05.410600 (+ 131us) server_negotiation.cc:299] Negotiation successful
0806 18:35:05.410836 (+ 236us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":308,"thread_start_us":122,"threads_started":1}
I20260806 18:35:05.412549 29844 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 9673a668f18b4e298809f6afecc58f4a: Neither blocks nor log segments found. Creating new log.
I20260806 18:35:05.413275 29844 log.cc:824] T 00000000000000000000000000000000 P 9673a668f18b4e298809f6afecc58f4a: Log is configured to *not* fsync() on all Append() calls
I20260806 18:35:05.414958 29844 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9673a668f18b4e298809f6afecc58f4a: No bootstrap required, opened a new log
I20260806 18:35:05.418165 29844 raft_consensus.cc:348] T 00000000000000000000000000000000 P 9673a668f18b4e298809f6afecc58f4a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9673a668f18b4e298809f6afecc58f4a" member_type: VOTER last_known_addr { host: "127.19.194.62" port: 34005 } }
I20260806 18:35:05.418359 29844 raft_consensus.cc:374] T 00000000000000000000000000000000 P 9673a668f18b4e298809f6afecc58f4a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 18:35:05.418455 29844 raft_consensus.cc:729] T 00000000000000000000000000000000 P 9673a668f18b4e298809f6afecc58f4a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9673a668f18b4e298809f6afecc58f4a, State: Initialized, Role: FOLLOWER
I20260806 18:35:05.418887 29844 consensus_queue.cc:260] T 00000000000000000000000000000000 P 9673a668f18b4e298809f6afecc58f4a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9673a668f18b4e298809f6afecc58f4a" member_type: VOTER last_known_addr { host: "127.19.194.62" port: 34005 } }
I20260806 18:35:05.419023 29844 raft_consensus.cc:388] T 00000000000000000000000000000000 P 9673a668f18b4e298809f6afecc58f4a [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 18:35:05.419086 29844 raft_consensus.cc:482] T 00000000000000000000000000000000 P 9673a668f18b4e298809f6afecc58f4a [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 18:35:05.419242 29844 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 9673a668f18b4e298809f6afecc58f4a [term 0 FOLLOWER]: Advancing to term 1
I20260806 18:35:05.420032 29844 raft_consensus.cc:504] T 00000000000000000000000000000000 P 9673a668f18b4e298809f6afecc58f4a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9673a668f18b4e298809f6afecc58f4a" member_type: VOTER last_known_addr { host: "127.19.194.62" port: 34005 } }
I20260806 18:35:05.420375 29844 leader_election.cc:302] T 00000000000000000000000000000000 P 9673a668f18b4e298809f6afecc58f4a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 9673a668f18b4e298809f6afecc58f4a; no voters:
I20260806 18:35:05.420650 29844 leader_election.cc:288] T 00000000000000000000000000000000 P 9673a668f18b4e298809f6afecc58f4a [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 18:35:05.420779 29849 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 9673a668f18b4e298809f6afecc58f4a [term 1 FOLLOWER]: Leader election won for term 1
I20260806 18:35:05.421101 29849 raft_consensus.cc:686] T 00000000000000000000000000000000 P 9673a668f18b4e298809f6afecc58f4a [term 1 LEADER]: Becoming Leader. State: Replica: 9673a668f18b4e298809f6afecc58f4a, State: Running, Role: LEADER
I20260806 18:35:05.421392 29849 consensus_queue.cc:237] T 00000000000000000000000000000000 P 9673a668f18b4e298809f6afecc58f4a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9673a668f18b4e298809f6afecc58f4a" member_type: VOTER last_known_addr { host: "127.19.194.62" port: 34005 } }
I20260806 18:35:05.421833 29844 sys_catalog.cc:564] T 00000000000000000000000000000000 P 9673a668f18b4e298809f6afecc58f4a [sys.catalog]: configured and running, proceeding with master startup.
I20260806 18:35:05.423344 29850 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9673a668f18b4e298809f6afecc58f4a [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "9673a668f18b4e298809f6afecc58f4a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9673a668f18b4e298809f6afecc58f4a" member_type: VOTER last_known_addr { host: "127.19.194.62" port: 34005 } } }
I20260806 18:35:05.423354 29851 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9673a668f18b4e298809f6afecc58f4a [sys.catalog]: SysCatalogTable state changed. Reason: New leader 9673a668f18b4e298809f6afecc58f4a. Latest consensus state: current_term: 1 leader_uuid: "9673a668f18b4e298809f6afecc58f4a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9673a668f18b4e298809f6afecc58f4a" member_type: VOTER last_known_addr { host: "127.19.194.62" port: 34005 } } }
I20260806 18:35:05.423493 29851 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9673a668f18b4e298809f6afecc58f4a [sys.catalog]: This master's current role is: LEADER
I20260806 18:35:05.423493 29850 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9673a668f18b4e298809f6afecc58f4a [sys.catalog]: This master's current role is: LEADER
I20260806 18:35:05.424275 29858 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 18:35:05.426749 29858 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 18:35:05.431453 29858 catalog_manager.cc:1347] Generated new cluster ID: 62080739a0e0407b8350cb55a722f08a
I20260806 18:35:05.431516 29858 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 18:35:05.444681 29858 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 18:35:05.445489 29858 catalog_manager.cc:1504] Loading token signing keys...
I20260806 18:35:05.454941 29858 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 9673a668f18b4e298809f6afecc58f4a: Generated new TSK 0
I20260806 18:35:05.455622 29858 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 18:35:05.463806 20232 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJW69tn/build/debug/bin/kudu
/tmp/dist-test-taskJW69tn/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786041246094603-20232-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786041246094603-20232-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786041246094603-20232-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786041246094603-20232-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.19.194.1:0
--local_ip_for_outbound_sockets=127.19.194.1
--webserver_interface=127.19.194.1
--webserver_port=0
--tserver_master_addrs=127.19.194.62:34005
--builtin_ntp_servers=127.19.194.20:38573
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
W20260806 18:35:05.577817 29868 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 18:35:05.578056 29868 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 18:35:05.578094 29868 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 18:35:05.582195 29868 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 18:35:05.582247 29868 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 18:35:05.582345 29868 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.19.194.1
I20260806 18:35:05.586903 29868 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.194.20:38573
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786041246094603-20232-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786041246094603-20232-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.19.194.1:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786041246094603-20232-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.19.194.1
--webserver_port=0
--tserver_master_addrs=127.19.194.62:34005
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29868
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.19.194.1
--log_dir=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786041246094603-20232-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 17:59:13 UTC on ccycloud.quasar-wvsnzx.root.comops.site
build id 4571028
I20260806 18:35:05.587898 29868 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 18:35:05.589322 29868 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 18:35:05.598598 29873 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:35:05.598594 29874 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:35:05.598639 29876 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 18:35:05.598783 29868 server_base.cc:1034] running on GCE node
I20260806 18:35:05.599291 29868 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 18:35:05.600188 29868 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 18:35:05.601413 29868 hybrid_clock.cc:648] HybridClock initialized: now 1786041305601403 us; error 44 us; skew 500 ppm
I20260806 18:35:05.603247 29868 webserver.cc:460] Webserver started at http://127.19.194.1:40229/ using document root <none> and password file <none>
I20260806 18:35:05.603822 29868 fs_manager.cc:359] Metadata directory not provided
I20260806 18:35:05.603921 29868 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 18:35:05.604149 29868 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 18:35:05.606185 29868 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786041246094603-20232-0/minicluster-data/ts-0/data/instance:
uuid: "c8fe19af6dda4aa292374498b82f6a3f"
format_stamp: "Formatted at 2026-08-06 18:35:05 on dist-test-slave-1xrh"
server_key: "911e563bdad604f07c4745a89ee52b22"
server_key_iv: "9107bf5ff601cd86e199bb5c2c5d785f"
server_key_version: "encryptionkey@0"
I20260806 18:35:05.606745 29868 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786041246094603-20232-0/minicluster-data/ts-0/wal/instance:
uuid: "c8fe19af6dda4aa292374498b82f6a3f"
format_stamp: "Formatted at 2026-08-06 18:35:05 on dist-test-slave-1xrh"
server_key: "911e563bdad604f07c4745a89ee52b22"
server_key_iv: "9107bf5ff601cd86e199bb5c2c5d785f"
server_key_version: "encryptionkey@0"
I20260806 18:35:05.610352 29868 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.006s sys 0.000s
I20260806 18:35:05.612792 29882 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 18:35:05.613955 29868 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260806 18:35:05.614102 29868 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786041246094603-20232-0/minicluster-data/ts-0/data,/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786041246094603-20232-0/minicluster-data/ts-0/wal
uuid: "c8fe19af6dda4aa292374498b82f6a3f"
format_stamp: "Formatted at 2026-08-06 18:35:05 on dist-test-slave-1xrh"
server_key: "911e563bdad604f07c4745a89ee52b22"
server_key_iv: "9107bf5ff601cd86e199bb5c2c5d785f"
server_key_version: "encryptionkey@0"
I20260806 18:35:05.614217 29868 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786041246094603-20232-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786041246094603-20232-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786041246094603-20232-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 18:35:05.633214 29868 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 18:35:05.633934 29868 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 18:35:05.634135 29868 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 18:35:05.634701 29868 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 18:35:05.635701 29868 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 18:35:05.635774 29868 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 18:35:05.635854 29868 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 18:35:05.635907 29868 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 18:35:05.645172 29868 rpc_server.cc:307] RPC server started. Bound to: 127.19.194.1:46075
I20260806 18:35:05.645205 29997 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.194.1:46075 every 8 connection(s)
I20260806 18:35:05.646025 29868 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786041246094603-20232-0/minicluster-data/ts-0/data/info.pb
I20260806 18:35:05.650580 20232 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJW69tn/build/debug/bin/kudu as pid 29868
I20260806 18:35:05.650707 20232 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786041246094603-20232-0/minicluster-data/ts-0/wal/instance
I20260806 18:35:05.651000 20232 external_mini_cluster.cc:1428] Setting key bb347c11f0fc2eda566d6f82b4cf0108
I20260806 18:35:05.653031 20232 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJW69tn/build/debug/bin/kudu
/tmp/dist-test-taskJW69tn/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786041246094603-20232-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786041246094603-20232-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786041246094603-20232-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786041246094603-20232-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.19.194.2:0
--local_ip_for_outbound_sockets=127.19.194.2
--webserver_interface=127.19.194.2
--webserver_port=0
--tserver_master_addrs=127.19.194.62:34005
--builtin_ntp_servers=127.19.194.20:38573
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
I20260806 18:35:05.655867 29847 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:35:05.647836 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.194.1:44105 (local address 127.19.194.62:34005)
0806 18:35:05.647981 (+ 145us) server_negotiation.cc:206] Beginning negotiation
0806 18:35:05.647985 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 18:35:05.649018 (+ 1033us) server_negotiation.cc:407] Connection header received
0806 18:35:05.650461 (+ 1443us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:35:05.650465 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:35:05.650529 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:35:05.650634 (+ 105us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:35:05.651720 (+ 1086us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:35:05.652441 (+ 721us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:35:05.653697 (+ 1256us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:35:05.654105 (+ 408us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:35:05.654235 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:35:05.655384 (+ 1149us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:35:05.655387 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:35:05.655389 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 18:35:05.655401 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:35:05.655421 (+ 20us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 18:35:05.655429 (+ 8us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:35:05.655490 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0806 18:35:05.655679 (+ 189us) server_negotiation.cc:299] Negotiation successful
0806 18:35:05.655756 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260806 18:35:05.656795 30000 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:35:05.648226 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.194.62:34005 (local address 127.19.194.1:44105)
0806 18:35:05.648865 (+ 639us) negotiation.cc:107] Waiting for socket to connect
0806 18:35:05.648905 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0806 18:35:05.650245 (+ 1340us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:35:05.650841 (+ 596us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:35:05.650848 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:35:05.650930 (+ 82us) client_negotiation.cc:189] Negotiated authn=SASL
0806 18:35:05.651556 (+ 626us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:35:05.651580 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:35:05.652571 (+ 991us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:35:05.652575 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:35:05.653572 (+ 997us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:35:05.653583 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:35:05.654260 (+ 677us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:35:05.654263 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:35:05.654399 (+ 136us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:35:05.655055 (+ 656us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0806 18:35:05.655059 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0806 18:35:05.655113 (+ 54us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0806 18:35:05.655224 (+ 111us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 18:35:05.655522 (+ 298us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 18:35:05.655532 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 18:35:05.655538 (+ 6us) client_negotiation.cc:769] Sending connection context
0806 18:35:05.655677 (+ 139us) client_negotiation.cc:240] Negotiation successful
0806 18:35:05.656006 (+ 329us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":258,"thread_start_us":111,"threads_started":1}
I20260806 18:35:05.658015 29998 heartbeater.cc:347] Connected to a master server at 127.19.194.62:34005
I20260806 18:35:05.658283 29998 heartbeater.cc:464] Registering TS with master...
I20260806 18:35:05.658917 29998 heartbeater.cc:511] Master 127.19.194.62:34005 requested a full tablet report, sending...
I20260806 18:35:05.660612 29808 ts_manager.cc:194] Registered new tserver with Master: c8fe19af6dda4aa292374498b82f6a3f (127.19.194.1:46075)
I20260806 18:35:05.661764 29808 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.19.194.1:44105
W20260806 18:35:05.762495 30001 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 18:35:05.762722 30001 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 18:35:05.762774 30001 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 18:35:05.766844 30001 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 18:35:05.766894 30001 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 18:35:05.766975 30001 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.19.194.2
I20260806 18:35:05.771559 30001 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.194.20:38573
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786041246094603-20232-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786041246094603-20232-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.19.194.2:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786041246094603-20232-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.19.194.2
--webserver_port=0
--tserver_master_addrs=127.19.194.62:34005
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30001
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.19.194.2
--log_dir=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786041246094603-20232-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 17:59:13 UTC on ccycloud.quasar-wvsnzx.root.comops.site
build id 4571028
I20260806 18:35:05.772541 30001 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 18:35:05.773826 30001 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260806 18:35:05.783172 30001 server_base.cc:1034] running on GCE node
W20260806 18:35:05.783068 30006 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:35:05.783082 30009 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:35:05.783066 30007 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 18:35:05.783802 30001 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 18:35:05.784467 30001 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 18:35:05.785624 30001 hybrid_clock.cc:648] HybridClock initialized: now 1786041305785594 us; error 42 us; skew 500 ppm
I20260806 18:35:05.787377 30001 webserver.cc:460] Webserver started at http://127.19.194.2:43671/ using document root <none> and password file <none>
I20260806 18:35:05.787943 30001 fs_manager.cc:359] Metadata directory not provided
I20260806 18:35:05.788005 30001 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 18:35:05.788262 30001 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 18:35:05.790306 30001 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786041246094603-20232-0/minicluster-data/ts-1/data/instance:
uuid: "c93f4e5fc4324fee909f32a387407832"
format_stamp: "Formatted at 2026-08-06 18:35:05 on dist-test-slave-1xrh"
server_key: "ac5289cfc1d7140cc69214a30f5dfe9b"
server_key_iv: "e31ec18531be59639b59b68abc800880"
server_key_version: "encryptionkey@0"
I20260806 18:35:05.790839 30001 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786041246094603-20232-0/minicluster-data/ts-1/wal/instance:
uuid: "c93f4e5fc4324fee909f32a387407832"
format_stamp: "Formatted at 2026-08-06 18:35:05 on dist-test-slave-1xrh"
server_key: "ac5289cfc1d7140cc69214a30f5dfe9b"
server_key_iv: "e31ec18531be59639b59b68abc800880"
server_key_version: "encryptionkey@0"
I20260806 18:35:05.794410 30001 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.006s sys 0.001s
I20260806 18:35:05.797005 30015 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 18:35:05.798177 30001 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260806 18:35:05.798312 30001 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786041246094603-20232-0/minicluster-data/ts-1/data,/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786041246094603-20232-0/minicluster-data/ts-1/wal
uuid: "c93f4e5fc4324fee909f32a387407832"
format_stamp: "Formatted at 2026-08-06 18:35:05 on dist-test-slave-1xrh"
server_key: "ac5289cfc1d7140cc69214a30f5dfe9b"
server_key_iv: "e31ec18531be59639b59b68abc800880"
server_key_version: "encryptionkey@0"
I20260806 18:35:05.798413 30001 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786041246094603-20232-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786041246094603-20232-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786041246094603-20232-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 18:35:05.811873 30001 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 18:35:05.812492 30001 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 18:35:05.812667 30001 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 18:35:05.813201 30001 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 18:35:05.814119 30001 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 18:35:05.814167 30001 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 18:35:05.814234 30001 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 18:35:05.814284 30001 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 18:35:05.823444 30001 rpc_server.cc:307] RPC server started. Bound to: 127.19.194.2:40737
I20260806 18:35:05.823513 30130 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.194.2:40737 every 8 connection(s)
I20260806 18:35:05.824354 30001 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786041246094603-20232-0/minicluster-data/ts-1/data/info.pb
I20260806 18:35:05.829883 20232 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJW69tn/build/debug/bin/kudu as pid 30001
I20260806 18:35:05.830010 20232 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786041246094603-20232-0/minicluster-data/ts-1/wal/instance
I20260806 18:35:05.830313 20232 external_mini_cluster.cc:1428] Setting key 8678a3e5ebfd3e26ecb83e892577d4b1
I20260806 18:35:05.832237 20232 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJW69tn/build/debug/bin/kudu
/tmp/dist-test-taskJW69tn/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786041246094603-20232-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786041246094603-20232-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786041246094603-20232-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786041246094603-20232-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.19.194.3:0
--local_ip_for_outbound_sockets=127.19.194.3
--webserver_interface=127.19.194.3
--webserver_port=0
--tserver_master_addrs=127.19.194.62:34005
--builtin_ntp_servers=127.19.194.20:38573
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
I20260806 18:35:05.833501 29847 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:35:05.826143 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.194.2:44437 (local address 127.19.194.62:34005)
0806 18:35:05.826286 (+ 143us) server_negotiation.cc:206] Beginning negotiation
0806 18:35:05.826290 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 18:35:05.827337 (+ 1047us) server_negotiation.cc:407] Connection header received
0806 18:35:05.828627 (+ 1290us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:35:05.828630 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:35:05.828678 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:35:05.828752 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:35:05.829556 (+ 804us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:35:05.830380 (+ 824us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:35:05.831558 (+ 1178us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:35:05.831798 (+ 240us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:35:05.831913 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:35:05.832991 (+ 1078us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:35:05.832994 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:35:05.832996 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 18:35:05.833008 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:35:05.833028 (+ 20us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 18:35:05.833036 (+ 8us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:35:05.833113 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0806 18:35:05.833258 (+ 145us) server_negotiation.cc:299] Negotiation successful
0806 18:35:05.833323 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260806 18:35:05.834343 30133 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:35:05.826526 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.194.62:34005 (local address 127.19.194.2:44437)
0806 18:35:05.827195 (+ 669us) negotiation.cc:107] Waiting for socket to connect
0806 18:35:05.827234 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0806 18:35:05.828445 (+ 1211us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:35:05.828904 (+ 459us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:35:05.828910 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:35:05.828986 (+ 76us) client_negotiation.cc:189] Negotiated authn=SASL
0806 18:35:05.829435 (+ 449us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:35:05.829447 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:35:05.830517 (+ 1070us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:35:05.830521 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:35:05.831420 (+ 899us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:35:05.831430 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:35:05.831951 (+ 521us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:35:05.831956 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:35:05.832113 (+ 157us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:35:05.832735 (+ 622us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0806 18:35:05.832737 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0806 18:35:05.832773 (+ 36us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0806 18:35:05.832831 (+ 58us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 18:35:05.833117 (+ 286us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 18:35:05.833129 (+ 12us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 18:35:05.833135 (+ 6us) client_negotiation.cc:769] Sending connection context
0806 18:35:05.833257 (+ 122us) client_negotiation.cc:240] Negotiation successful
0806 18:35:05.833523 (+ 266us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":271,"thread_start_us":114,"threads_started":1}
I20260806 18:35:05.835506 30131 heartbeater.cc:347] Connected to a master server at 127.19.194.62:34005
I20260806 18:35:05.835732 30131 heartbeater.cc:464] Registering TS with master...
I20260806 18:35:05.836330 30131 heartbeater.cc:511] Master 127.19.194.62:34005 requested a full tablet report, sending...
I20260806 18:35:05.837443 29808 ts_manager.cc:194] Registered new tserver with Master: c93f4e5fc4324fee909f32a387407832 (127.19.194.2:40737)
I20260806 18:35:05.838009 29808 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.19.194.2:44437
W20260806 18:35:05.950030 30134 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 18:35:05.950280 30134 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 18:35:05.950367 30134 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 18:35:05.955018 30134 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 18:35:05.955113 30134 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 18:35:05.955265 30134 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.19.194.3
I20260806 18:35:05.960670 30134 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.194.20:38573
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786041246094603-20232-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786041246094603-20232-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.19.194.3:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786041246094603-20232-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.19.194.3
--webserver_port=0
--tserver_master_addrs=127.19.194.62:34005
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30134
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.19.194.3
--log_dir=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786041246094603-20232-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 17:59:13 UTC on ccycloud.quasar-wvsnzx.root.comops.site
build id 4571028
I20260806 18:35:05.961751 30134 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 18:35:05.963200 30134 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 18:35:05.973105 30139 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 18:35:05.973227 30134 server_base.cc:1034] running on GCE node
W20260806 18:35:05.973188 30142 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:35:05.973162 30140 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 18:35:05.973738 30134 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 18:35:05.974345 30134 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 18:35:05.975525 30134 hybrid_clock.cc:648] HybridClock initialized: now 1786041305975507 us; error 43 us; skew 500 ppm
I20260806 18:35:05.977195 30134 webserver.cc:460] Webserver started at http://127.19.194.3:34201/ using document root <none> and password file <none>
I20260806 18:35:05.977762 30134 fs_manager.cc:359] Metadata directory not provided
I20260806 18:35:05.977854 30134 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 18:35:05.978085 30134 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 18:35:05.980278 30134 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786041246094603-20232-0/minicluster-data/ts-2/data/instance:
uuid: "ba165de6b1bd44e4bd385daccd3aff07"
format_stamp: "Formatted at 2026-08-06 18:35:05 on dist-test-slave-1xrh"
server_key: "5d524c80d18eea4b76b1229bd7af95ef"
server_key_iv: "6389583b834d5b010edfc8a7831e2dbb"
server_key_version: "encryptionkey@0"
I20260806 18:35:05.980849 30134 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786041246094603-20232-0/minicluster-data/ts-2/wal/instance:
uuid: "ba165de6b1bd44e4bd385daccd3aff07"
format_stamp: "Formatted at 2026-08-06 18:35:05 on dist-test-slave-1xrh"
server_key: "5d524c80d18eea4b76b1229bd7af95ef"
server_key_iv: "6389583b834d5b010edfc8a7831e2dbb"
server_key_version: "encryptionkey@0"
I20260806 18:35:05.984445 30134 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20260806 18:35:05.986768 30148 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 18:35:05.988014 30134 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260806 18:35:05.988145 30134 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786041246094603-20232-0/minicluster-data/ts-2/data,/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786041246094603-20232-0/minicluster-data/ts-2/wal
uuid: "ba165de6b1bd44e4bd385daccd3aff07"
format_stamp: "Formatted at 2026-08-06 18:35:05 on dist-test-slave-1xrh"
server_key: "5d524c80d18eea4b76b1229bd7af95ef"
server_key_iv: "6389583b834d5b010edfc8a7831e2dbb"
server_key_version: "encryptionkey@0"
I20260806 18:35:05.988255 30134 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786041246094603-20232-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786041246094603-20232-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786041246094603-20232-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 18:35:06.007217 30134 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 18:35:06.007840 30134 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 18:35:06.008028 30134 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 18:35:06.008572 30134 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 18:35:06.009495 30134 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 18:35:06.009564 30134 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 18:35:06.009629 30134 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 18:35:06.009663 30134 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 18:35:06.018352 30134 rpc_server.cc:307] RPC server started. Bound to: 127.19.194.3:38627
I20260806 18:35:06.018378 30262 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.194.3:38627 every 8 connection(s)
I20260806 18:35:06.019378 30134 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786041246094603-20232-0/minicluster-data/ts-2/data/info.pb
I20260806 18:35:06.028537 29847 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:35:06.021142 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.194.3:40545 (local address 127.19.194.62:34005)
0806 18:35:06.021279 (+ 137us) server_negotiation.cc:206] Beginning negotiation
0806 18:35:06.021282 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 18:35:06.022320 (+ 1038us) server_negotiation.cc:407] Connection header received
0806 18:35:06.023894 (+ 1574us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:35:06.023898 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:35:06.023960 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:35:06.024076 (+ 116us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:35:06.024939 (+ 863us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:35:06.025475 (+ 536us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:35:06.026611 (+ 1136us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:35:06.026841 (+ 230us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:35:06.026957 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:35:06.028025 (+ 1068us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:35:06.028029 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:35:06.028032 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 18:35:06.028046 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:35:06.028072 (+ 26us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 18:35:06.028081 (+ 9us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:35:06.028161 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0806 18:35:06.028321 (+ 160us) server_negotiation.cc:299] Negotiation successful
0806 18:35:06.028382 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260806 18:35:06.029260 20232 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJW69tn/build/debug/bin/kudu as pid 30134
I20260806 18:35:06.029284 30265 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:35:06.021438 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.194.62:34005 (local address 127.19.194.3:40545)
0806 18:35:06.022175 (+ 737us) negotiation.cc:107] Waiting for socket to connect
0806 18:35:06.022214 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0806 18:35:06.023669 (+ 1455us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:35:06.024253 (+ 584us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:35:06.024261 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:35:06.024340 (+ 79us) client_negotiation.cc:189] Negotiated authn=SASL
0806 18:35:06.024786 (+ 446us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:35:06.024798 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:35:06.025620 (+ 822us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:35:06.025623 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:35:06.026479 (+ 856us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:35:06.026488 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:35:06.026998 (+ 510us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:35:06.027003 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:35:06.027229 (+ 226us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:35:06.027772 (+ 543us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0806 18:35:06.027774 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0806 18:35:06.027810 (+ 36us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0806 18:35:06.027870 (+ 60us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 18:35:06.028163 (+ 293us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 18:35:06.028176 (+ 13us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 18:35:06.028184 (+ 8us) client_negotiation.cc:769] Sending connection context
0806 18:35:06.028313 (+ 129us) client_negotiation.cc:240] Negotiation successful
0806 18:35:06.028572 (+ 259us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":308,"thread_start_us":152,"threads_started":1}
I20260806 18:35:06.029383 20232 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786041246094603-20232-0/minicluster-data/ts-2/wal/instance
I20260806 18:35:06.029698 20232 external_mini_cluster.cc:1428] Setting key 777866aafba4c0615c9b08b1fd85bfc5
I20260806 18:35:06.030541 30263 heartbeater.cc:347] Connected to a master server at 127.19.194.62:34005
I20260806 18:35:06.030772 30263 heartbeater.cc:464] Registering TS with master...
I20260806 18:35:06.031368 30263 heartbeater.cc:511] Master 127.19.194.62:34005 requested a full tablet report, sending...
I20260806 18:35:06.032373 29808 ts_manager.cc:194] Registered new tserver with Master: ba165de6b1bd44e4bd385daccd3aff07 (127.19.194.3:38627)
I20260806 18:35:06.032927 29808 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.19.194.3:40545
I20260806 18:35:06.043745 20232 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
W20260806 18:35:06.046409 29847 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 18:35:06.045605 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59396 (local address 127.19.194.62:34005)
0806 18:35:06.045733 (+ 128us) server_negotiation.cc:206] Beginning negotiation
0806 18:35:06.045736 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 18:35:06.045887 (+ 151us) server_negotiation.cc:407] Connection header received
0806 18:35:06.045963 (+ 76us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:35:06.045965 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:35:06.046000 (+ 35us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:35:06.046068 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:35:06.046277 (+ 209us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:59396: BlockingRecv error: recv got EOF from 127.0.0.1:59396 (error 108)
Metrics: {"server-negotiator.queue_time_us":48}
I20260806 18:35:06.046753 20232 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJW69tn/build/debug/bin/kudu with pid 29868
I20260806 18:35:06.056612 20232 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJW69tn/build/debug/bin/kudu with pid 30001
I20260806 18:35:06.064559 20232 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJW69tn/build/debug/bin/kudu with pid 30134
I20260806 18:35:06.070518 20232 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJW69tn/build/debug/bin/kudu with pid 29776
2026-08-06T18:35:06Z chronyd exiting
[ OK ] SecurityITest.TestRequireAuthenticationInsecureCluster (885 ms)
[ RUN ] SecurityITest.TestRequireEncryptionInsecureCluster
2026-08-06T18:35:06Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T18:35:06Z Disabled control of system clock
I20260806 18:35:06.101017 20232 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJW69tn/build/debug/bin/kudu
/tmp/dist-test-taskJW69tn/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786041246094603-20232-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786041246094603-20232-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786041246094603-20232-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786041246094603-20232-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.19.194.62:43071
--webserver_interface=127.19.194.62
--webserver_port=0
--builtin_ntp_servers=127.19.194.20:43749
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.19.194.62:43071
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
W20260806 18:35:06.208298 30274 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 18:35:06.208542 30274 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 18:35:06.208580 30274 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 18:35:06.212461 30274 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 18:35:06.212502 30274 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 18:35:06.212517 30274 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 18:35:06.212533 30274 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 18:35:06.212548 30274 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 18:35:06.216923 30274 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.194.20:43749
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786041246094603-20232-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786041246094603-20232-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.19.194.62:43071
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.19.194.62:43071
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786041246094603-20232-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.19.194.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30274
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786041246094603-20232-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 17:59:13 UTC on ccycloud.quasar-wvsnzx.root.comops.site
build id 4571028
I20260806 18:35:06.217836 30274 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 18:35:06.219069 30274 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 18:35:06.227300 30282 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:35:06.227305 30279 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:35:06.227315 30280 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 18:35:06.227360 30274 server_base.cc:1034] running on GCE node
I20260806 18:35:06.228034 30274 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 18:35:06.229097 30274 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 18:35:06.230260 30274 hybrid_clock.cc:648] HybridClock initialized: now 1786041306230234 us; error 39 us; skew 500 ppm
I20260806 18:35:06.232108 30274 webserver.cc:460] Webserver started at http://127.19.194.62:38265/ using document root <none> and password file <none>
I20260806 18:35:06.232666 30274 fs_manager.cc:359] Metadata directory not provided
I20260806 18:35:06.232728 30274 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 18:35:06.232973 30274 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 18:35:06.234901 30274 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786041246094603-20232-0/minicluster-data/master-0/data/instance:
uuid: "7e4bd34705f04dd3b95f7660c2a070ac"
format_stamp: "Formatted at 2026-08-06 18:35:06 on dist-test-slave-1xrh"
server_key: "22906def9929e8aed27ea925e4653eb5"
server_key_iv: "9d64cbc3b3d660a13ef2cab272f33d5b"
server_key_version: "encryptionkey@0"
I20260806 18:35:06.235522 30274 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786041246094603-20232-0/minicluster-data/master-0/wal/instance:
uuid: "7e4bd34705f04dd3b95f7660c2a070ac"
format_stamp: "Formatted at 2026-08-06 18:35:06 on dist-test-slave-1xrh"
server_key: "22906def9929e8aed27ea925e4653eb5"
server_key_iv: "9d64cbc3b3d660a13ef2cab272f33d5b"
server_key_version: "encryptionkey@0"
I20260806 18:35:06.238947 30274 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.000s
I20260806 18:35:06.241350 30288 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 18:35:06.242503 30274 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260806 18:35:06.242645 30274 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786041246094603-20232-0/minicluster-data/master-0/data,/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786041246094603-20232-0/minicluster-data/master-0/wal
uuid: "7e4bd34705f04dd3b95f7660c2a070ac"
format_stamp: "Formatted at 2026-08-06 18:35:06 on dist-test-slave-1xrh"
server_key: "22906def9929e8aed27ea925e4653eb5"
server_key_iv: "9d64cbc3b3d660a13ef2cab272f33d5b"
server_key_version: "encryptionkey@0"
I20260806 18:35:06.242755 30274 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786041246094603-20232-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786041246094603-20232-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786041246094603-20232-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 18:35:06.247296 30274 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 18:35:06.247886 30274 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 18:35:06.248077 30274 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 18:35:06.254760 30274 rpc_server.cc:307] RPC server started. Bound to: 127.19.194.62:43071
I20260806 18:35:06.254768 30340 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.194.62:43071 every 8 connection(s)
I20260806 18:35:06.255661 30274 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786041246094603-20232-0/minicluster-data/master-0/data/info.pb
I20260806 18:35:06.256249 20232 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJW69tn/build/debug/bin/kudu as pid 30274
I20260806 18:35:06.256386 20232 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786041246094603-20232-0/minicluster-data/master-0/wal/instance
I20260806 18:35:06.256664 20232 external_mini_cluster.cc:1428] Setting key 08ba47c5b303c284f854830fce4f149f
I20260806 18:35:06.258908 30341 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 18:35:06.262662 30344 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:35:06.258019 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42134 (local address 127.19.194.62:43071)
0806 18:35:06.259268 (+ 1249us) server_negotiation.cc:206] Beginning negotiation
0806 18:35:06.259285 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0806 18:35:06.259318 (+ 33us) server_negotiation.cc:407] Connection header received
0806 18:35:06.259940 (+ 622us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:35:06.259955 (+ 15us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:35:06.260282 (+ 327us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:35:06.260611 (+ 329us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:35:06.261445 (+ 834us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:35:06.261454 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:35:06.261470 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 18:35:06.261486 (+ 16us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:35:06.261559 (+ 73us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 18:35:06.261584 (+ 25us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:35:06.261742 (+ 158us) server_negotiation.cc:1035] Waiting for connection context
0806 18:35:06.261812 (+ 70us) server_negotiation.cc:299] Negotiation successful
0806 18:35:06.262045 (+ 233us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":816,"thread_start_us":656,"threads_started":1}
I20260806 18:35:06.265151 30341 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7e4bd34705f04dd3b95f7660c2a070ac: Bootstrap starting.
I20260806 18:35:06.267194 30341 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 7e4bd34705f04dd3b95f7660c2a070ac: Neither blocks nor log segments found. Creating new log.
I20260806 18:35:06.267911 30341 log.cc:824] T 00000000000000000000000000000000 P 7e4bd34705f04dd3b95f7660c2a070ac: Log is configured to *not* fsync() on all Append() calls
I20260806 18:35:06.269925 30341 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7e4bd34705f04dd3b95f7660c2a070ac: No bootstrap required, opened a new log
I20260806 18:35:06.273209 30341 raft_consensus.cc:348] T 00000000000000000000000000000000 P 7e4bd34705f04dd3b95f7660c2a070ac [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7e4bd34705f04dd3b95f7660c2a070ac" member_type: VOTER last_known_addr { host: "127.19.194.62" port: 43071 } }
I20260806 18:35:06.273433 30341 raft_consensus.cc:374] T 00000000000000000000000000000000 P 7e4bd34705f04dd3b95f7660c2a070ac [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 18:35:06.273543 30341 raft_consensus.cc:729] T 00000000000000000000000000000000 P 7e4bd34705f04dd3b95f7660c2a070ac [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7e4bd34705f04dd3b95f7660c2a070ac, State: Initialized, Role: FOLLOWER
I20260806 18:35:06.273936 30341 consensus_queue.cc:260] T 00000000000000000000000000000000 P 7e4bd34705f04dd3b95f7660c2a070ac [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7e4bd34705f04dd3b95f7660c2a070ac" member_type: VOTER last_known_addr { host: "127.19.194.62" port: 43071 } }
I20260806 18:35:06.274091 30341 raft_consensus.cc:388] T 00000000000000000000000000000000 P 7e4bd34705f04dd3b95f7660c2a070ac [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 18:35:06.274185 30341 raft_consensus.cc:482] T 00000000000000000000000000000000 P 7e4bd34705f04dd3b95f7660c2a070ac [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 18:35:06.274317 30341 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 7e4bd34705f04dd3b95f7660c2a070ac [term 0 FOLLOWER]: Advancing to term 1
I20260806 18:35:06.275115 30341 raft_consensus.cc:504] T 00000000000000000000000000000000 P 7e4bd34705f04dd3b95f7660c2a070ac [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7e4bd34705f04dd3b95f7660c2a070ac" member_type: VOTER last_known_addr { host: "127.19.194.62" port: 43071 } }
I20260806 18:35:06.275563 30341 leader_election.cc:302] T 00000000000000000000000000000000 P 7e4bd34705f04dd3b95f7660c2a070ac [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 7e4bd34705f04dd3b95f7660c2a070ac; no voters:
I20260806 18:35:06.275884 30341 leader_election.cc:288] T 00000000000000000000000000000000 P 7e4bd34705f04dd3b95f7660c2a070ac [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 18:35:06.276160 30346 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 7e4bd34705f04dd3b95f7660c2a070ac [term 1 FOLLOWER]: Leader election won for term 1
I20260806 18:35:06.276427 30346 raft_consensus.cc:686] T 00000000000000000000000000000000 P 7e4bd34705f04dd3b95f7660c2a070ac [term 1 LEADER]: Becoming Leader. State: Replica: 7e4bd34705f04dd3b95f7660c2a070ac, State: Running, Role: LEADER
I20260806 18:35:06.276752 30346 consensus_queue.cc:237] T 00000000000000000000000000000000 P 7e4bd34705f04dd3b95f7660c2a070ac [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7e4bd34705f04dd3b95f7660c2a070ac" member_type: VOTER last_known_addr { host: "127.19.194.62" port: 43071 } }
I20260806 18:35:06.277135 30341 sys_catalog.cc:564] T 00000000000000000000000000000000 P 7e4bd34705f04dd3b95f7660c2a070ac [sys.catalog]: configured and running, proceeding with master startup.
I20260806 18:35:06.278352 30347 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7e4bd34705f04dd3b95f7660c2a070ac [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "7e4bd34705f04dd3b95f7660c2a070ac" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7e4bd34705f04dd3b95f7660c2a070ac" member_type: VOTER last_known_addr { host: "127.19.194.62" port: 43071 } } }
I20260806 18:35:06.278491 30347 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7e4bd34705f04dd3b95f7660c2a070ac [sys.catalog]: This master's current role is: LEADER
I20260806 18:35:06.278723 30348 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7e4bd34705f04dd3b95f7660c2a070ac [sys.catalog]: SysCatalogTable state changed. Reason: New leader 7e4bd34705f04dd3b95f7660c2a070ac. Latest consensus state: current_term: 1 leader_uuid: "7e4bd34705f04dd3b95f7660c2a070ac" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7e4bd34705f04dd3b95f7660c2a070ac" member_type: VOTER last_known_addr { host: "127.19.194.62" port: 43071 } } }
I20260806 18:35:06.278868 30348 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7e4bd34705f04dd3b95f7660c2a070ac [sys.catalog]: This master's current role is: LEADER
I20260806 18:35:06.278888 30356 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 18:35:06.281819 30356 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 18:35:06.287101 30356 catalog_manager.cc:1347] Generated new cluster ID: 05b2f4415f834794a2cde1079269f832
I20260806 18:35:06.287202 30356 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 18:35:06.303697 30356 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 18:35:06.304059 30356 catalog_manager.cc:1504] Loading token signing keys...
I20260806 18:35:06.311347 30356 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 7e4bd34705f04dd3b95f7660c2a070ac: Generated new TSK 0
I20260806 18:35:06.311969 30356 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 18:35:06.320150 20232 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJW69tn/build/debug/bin/kudu
/tmp/dist-test-taskJW69tn/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786041246094603-20232-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786041246094603-20232-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786041246094603-20232-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786041246094603-20232-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.19.194.1:0
--local_ip_for_outbound_sockets=127.19.194.1
--webserver_interface=127.19.194.1
--webserver_port=0
--tserver_master_addrs=127.19.194.62:43071
--builtin_ntp_servers=127.19.194.20:43749
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
W20260806 18:35:06.439261 30366 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 18:35:06.439482 30366 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 18:35:06.439518 30366 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 18:35:06.443225 30366 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 18:35:06.443265 30366 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 18:35:06.443354 30366 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.19.194.1
I20260806 18:35:06.448137 30366 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.194.20:43749
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786041246094603-20232-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786041246094603-20232-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.19.194.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786041246094603-20232-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.19.194.1
--webserver_port=0
--tserver_master_addrs=127.19.194.62:43071
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30366
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.19.194.1
--log_dir=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786041246094603-20232-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 17:59:13 UTC on ccycloud.quasar-wvsnzx.root.comops.site
build id 4571028
I20260806 18:35:06.449082 30366 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 18:35:06.450366 30366 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 18:35:06.459407 30371 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:35:06.459412 30372 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:35:06.459422 30374 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 18:35:06.459667 30366 server_base.cc:1034] running on GCE node
I20260806 18:35:06.460096 30366 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 18:35:06.460660 30366 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 18:35:06.461819 30366 hybrid_clock.cc:648] HybridClock initialized: now 1786041306461802 us; error 28 us; skew 500 ppm
I20260806 18:35:06.463539 30366 webserver.cc:460] Webserver started at http://127.19.194.1:44715/ using document root <none> and password file <none>
I20260806 18:35:06.464120 30366 fs_manager.cc:359] Metadata directory not provided
I20260806 18:35:06.464210 30366 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 18:35:06.464433 30366 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 18:35:06.466449 30366 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786041246094603-20232-0/minicluster-data/ts-0/data/instance:
uuid: "338dc7865baf41168ae78abdf32b0ba2"
format_stamp: "Formatted at 2026-08-06 18:35:06 on dist-test-slave-1xrh"
server_key: "6e696cc4af3fbfea468370eec947e61c"
server_key_iv: "55d02bf7044c9e696badad5c52bd1113"
server_key_version: "encryptionkey@0"
I20260806 18:35:06.467058 30366 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786041246094603-20232-0/minicluster-data/ts-0/wal/instance:
uuid: "338dc7865baf41168ae78abdf32b0ba2"
format_stamp: "Formatted at 2026-08-06 18:35:06 on dist-test-slave-1xrh"
server_key: "6e696cc4af3fbfea468370eec947e61c"
server_key_iv: "55d02bf7044c9e696badad5c52bd1113"
server_key_version: "encryptionkey@0"
I20260806 18:35:06.470626 30366 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.002s sys 0.000s
I20260806 18:35:06.473070 30380 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 18:35:06.474185 30366 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.000s sys 0.002s
I20260806 18:35:06.474313 30366 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786041246094603-20232-0/minicluster-data/ts-0/data,/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786041246094603-20232-0/minicluster-data/ts-0/wal
uuid: "338dc7865baf41168ae78abdf32b0ba2"
format_stamp: "Formatted at 2026-08-06 18:35:06 on dist-test-slave-1xrh"
server_key: "6e696cc4af3fbfea468370eec947e61c"
server_key_iv: "55d02bf7044c9e696badad5c52bd1113"
server_key_version: "encryptionkey@0"
I20260806 18:35:06.474422 30366 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786041246094603-20232-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786041246094603-20232-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786041246094603-20232-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 18:35:06.479017 30366 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 18:35:06.479653 30366 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 18:35:06.479825 30366 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 18:35:06.480370 30366 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 18:35:06.481200 30366 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 18:35:06.481244 30366 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 18:35:06.481325 30366 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 18:35:06.481357 30366 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 18:35:06.490239 30366 rpc_server.cc:307] RPC server started. Bound to: 127.19.194.1:40119
I20260806 18:35:06.490285 30497 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.194.1:40119 every 8 connection(s)
I20260806 18:35:06.491091 30366 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786041246094603-20232-0/minicluster-data/ts-0/data/info.pb
I20260806 18:35:06.496676 30344 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:35:06.492853 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.194.1:51519 (local address 127.19.194.62:43071)
0806 18:35:06.493004 (+ 151us) server_negotiation.cc:206] Beginning negotiation
0806 18:35:06.493009 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 18:35:06.494018 (+ 1009us) server_negotiation.cc:407] Connection header received
0806 18:35:06.495273 (+ 1255us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:35:06.495275 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:35:06.495310 (+ 35us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:35:06.495381 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:35:06.496286 (+ 905us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:35:06.496289 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:35:06.496291 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 18:35:06.496301 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:35:06.496319 (+ 18us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 18:35:06.496327 (+ 8us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:35:06.496425 (+ 98us) server_negotiation.cc:1035] Waiting for connection context
0806 18:35:06.496538 (+ 113us) server_negotiation.cc:299] Negotiation successful
0806 18:35:06.496573 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260806 18:35:06.497102 20232 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJW69tn/build/debug/bin/kudu as pid 30366
I20260806 18:35:06.497200 20232 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786041246094603-20232-0/minicluster-data/ts-0/wal/instance
I20260806 18:35:06.497510 20232 external_mini_cluster.cc:1428] Setting key 444346ee851595c06ca95ac4e36dcc36
I20260806 18:35:06.497622 30500 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:35:06.493281 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.194.62:43071 (local address 127.19.194.1:51519)
0806 18:35:06.493872 (+ 591us) negotiation.cc:107] Waiting for socket to connect
0806 18:35:06.493911 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0806 18:35:06.495028 (+ 1117us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:35:06.495530 (+ 502us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:35:06.495535 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:35:06.495611 (+ 76us) client_negotiation.cc:189] Negotiated authn=SASL
0806 18:35:06.496065 (+ 454us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0806 18:35:06.496066 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0806 18:35:06.496101 (+ 35us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0806 18:35:06.496162 (+ 61us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 18:35:06.496422 (+ 260us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 18:35:06.496429 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 18:35:06.496433 (+ 4us) client_negotiation.cc:769] Sending connection context
0806 18:35:06.496552 (+ 119us) client_negotiation.cc:240] Negotiation successful
0806 18:35:06.496721 (+ 169us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":236,"thread_start_us":105,"threads_started":1}
I20260806 18:35:06.498769 30498 heartbeater.cc:347] Connected to a master server at 127.19.194.62:43071
I20260806 18:35:06.498996 30498 heartbeater.cc:464] Registering TS with master...
I20260806 18:35:06.499423 30498 heartbeater.cc:511] Master 127.19.194.62:43071 requested a full tablet report, sending...
I20260806 18:35:06.499667 20232 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJW69tn/build/debug/bin/kudu
/tmp/dist-test-taskJW69tn/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786041246094603-20232-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786041246094603-20232-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786041246094603-20232-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786041246094603-20232-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.19.194.2:0
--local_ip_for_outbound_sockets=127.19.194.2
--webserver_interface=127.19.194.2
--webserver_port=0
--tserver_master_addrs=127.19.194.62:43071
--builtin_ntp_servers=127.19.194.20:43749
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
I20260806 18:35:06.500998 30304 ts_manager.cc:194] Registered new tserver with Master: 338dc7865baf41168ae78abdf32b0ba2 (127.19.194.1:40119)
W20260806 18:35:06.606288 30501 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 18:35:06.606594 30501 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 18:35:06.606668 30501 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 18:35:06.610592 30501 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 18:35:06.610668 30501 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 18:35:06.610754 30501 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.19.194.2
I20260806 18:35:06.615437 30501 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.194.20:43749
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786041246094603-20232-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786041246094603-20232-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.19.194.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786041246094603-20232-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.19.194.2
--webserver_port=0
--tserver_master_addrs=127.19.194.62:43071
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30501
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.19.194.2
--log_dir=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786041246094603-20232-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 17:59:13 UTC on ccycloud.quasar-wvsnzx.root.comops.site
build id 4571028
I20260806 18:35:06.616436 30501 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 18:35:06.617794 30501 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 18:35:06.627174 30507 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:35:06.627247 30506 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 18:35:06.627388 30501 server_base.cc:1034] running on GCE node
W20260806 18:35:06.627625 30509 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 18:35:06.628011 30501 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 18:35:06.628621 30501 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 18:35:06.629781 30501 hybrid_clock.cc:648] HybridClock initialized: now 1786041306629763 us; error 28 us; skew 500 ppm
I20260806 18:35:06.631551 30501 webserver.cc:460] Webserver started at http://127.19.194.2:40725/ using document root <none> and password file <none>
I20260806 18:35:06.632093 30501 fs_manager.cc:359] Metadata directory not provided
I20260806 18:35:06.632154 30501 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 18:35:06.632408 30501 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 18:35:06.634423 30501 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786041246094603-20232-0/minicluster-data/ts-1/data/instance:
uuid: "59e7f9f2165e469ca8328f11aaabd644"
format_stamp: "Formatted at 2026-08-06 18:35:06 on dist-test-slave-1xrh"
server_key: "999a4ec265cdae71fff55a26f99f6519"
server_key_iv: "4650917b14479733634a49248de7020d"
server_key_version: "encryptionkey@0"
I20260806 18:35:06.634972 30501 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786041246094603-20232-0/minicluster-data/ts-1/wal/instance:
uuid: "59e7f9f2165e469ca8328f11aaabd644"
format_stamp: "Formatted at 2026-08-06 18:35:06 on dist-test-slave-1xrh"
server_key: "999a4ec265cdae71fff55a26f99f6519"
server_key_iv: "4650917b14479733634a49248de7020d"
server_key_version: "encryptionkey@0"
I20260806 18:35:06.638429 30501 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.002s sys 0.000s
I20260806 18:35:06.640906 30516 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 18:35:06.642023 30501 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260806 18:35:06.642139 30501 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786041246094603-20232-0/minicluster-data/ts-1/data,/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786041246094603-20232-0/minicluster-data/ts-1/wal
uuid: "59e7f9f2165e469ca8328f11aaabd644"
format_stamp: "Formatted at 2026-08-06 18:35:06 on dist-test-slave-1xrh"
server_key: "999a4ec265cdae71fff55a26f99f6519"
server_key_iv: "4650917b14479733634a49248de7020d"
server_key_version: "encryptionkey@0"
I20260806 18:35:06.642236 30501 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786041246094603-20232-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786041246094603-20232-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786041246094603-20232-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 18:35:06.646716 30501 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 18:35:06.647414 30501 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 18:35:06.647584 30501 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 18:35:06.648116 30501 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 18:35:06.648967 30501 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 18:35:06.649010 30501 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 18:35:06.649073 30501 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 18:35:06.649104 30501 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 18:35:06.657598 30501 rpc_server.cc:307] RPC server started. Bound to: 127.19.194.2:33809
I20260806 18:35:06.657627 30632 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.194.2:33809 every 8 connection(s)
I20260806 18:35:06.658438 30501 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786041246094603-20232-0/minicluster-data/ts-1/data/info.pb
I20260806 18:35:06.664279 30344 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:35:06.660161 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.194.2:53803 (local address 127.19.194.62:43071)
0806 18:35:06.660293 (+ 132us) server_negotiation.cc:206] Beginning negotiation
0806 18:35:06.660296 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 18:35:06.661228 (+ 932us) server_negotiation.cc:407] Connection header received
0806 18:35:06.662535 (+ 1307us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:35:06.662538 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:35:06.662583 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:35:06.662680 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:35:06.663777 (+ 1097us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:35:06.663780 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:35:06.663793 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 18:35:06.663805 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:35:06.663851 (+ 46us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 18:35:06.663860 (+ 9us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:35:06.663964 (+ 104us) server_negotiation.cc:1035] Waiting for connection context
0806 18:35:06.664100 (+ 136us) server_negotiation.cc:299] Negotiation successful
0806 18:35:06.664147 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260806 18:35:06.665042 30635 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:35:06.660502 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.194.62:43071 (local address 127.19.194.2:53803)
0806 18:35:06.661092 (+ 590us) negotiation.cc:107] Waiting for socket to connect
0806 18:35:06.661128 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0806 18:35:06.662318 (+ 1190us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:35:06.662833 (+ 515us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:35:06.662839 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:35:06.662920 (+ 81us) client_negotiation.cc:189] Negotiated authn=SASL
0806 18:35:06.663513 (+ 593us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0806 18:35:06.663515 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0806 18:35:06.663552 (+ 37us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0806 18:35:06.663617 (+ 65us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 18:35:06.663948 (+ 331us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 18:35:06.663959 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 18:35:06.663964 (+ 5us) client_negotiation.cc:769] Sending connection context
0806 18:35:06.664141 (+ 177us) client_negotiation.cc:240] Negotiation successful
0806 18:35:06.664367 (+ 226us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":234,"thread_start_us":99,"threads_started":1}
I20260806 18:35:06.665938 20232 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJW69tn/build/debug/bin/kudu as pid 30501
I20260806 18:35:06.666069 20232 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786041246094603-20232-0/minicluster-data/ts-1/wal/instance
I20260806 18:35:06.666343 20232 external_mini_cluster.cc:1428] Setting key b3b064e84fe7845bd5df700cd3b54f33
I20260806 18:35:06.666342 30633 heartbeater.cc:347] Connected to a master server at 127.19.194.62:43071
I20260806 18:35:06.666561 30633 heartbeater.cc:464] Registering TS with master...
I20260806 18:35:06.666936 30633 heartbeater.cc:511] Master 127.19.194.62:43071 requested a full tablet report, sending...
I20260806 18:35:06.667959 30304 ts_manager.cc:194] Registered new tserver with Master: 59e7f9f2165e469ca8328f11aaabd644 (127.19.194.2:33809)
I20260806 18:35:06.668319 20232 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJW69tn/build/debug/bin/kudu
/tmp/dist-test-taskJW69tn/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786041246094603-20232-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786041246094603-20232-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786041246094603-20232-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786041246094603-20232-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.19.194.3:0
--local_ip_for_outbound_sockets=127.19.194.3
--webserver_interface=127.19.194.3
--webserver_port=0
--tserver_master_addrs=127.19.194.62:43071
--builtin_ntp_servers=127.19.194.20:43749
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
W20260806 18:35:06.775411 30636 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 18:35:06.775628 30636 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 18:35:06.775678 30636 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 18:35:06.779892 30636 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 18:35:06.779933 30636 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 18:35:06.780021 30636 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.19.194.3
I20260806 18:35:06.784775 30636 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.194.20:43749
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786041246094603-20232-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786041246094603-20232-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.19.194.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786041246094603-20232-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.19.194.3
--webserver_port=0
--tserver_master_addrs=127.19.194.62:43071
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30636
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.19.194.3
--log_dir=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786041246094603-20232-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 17:59:13 UTC on ccycloud.quasar-wvsnzx.root.comops.site
build id 4571028
I20260806 18:35:06.785706 30636 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 18:35:06.787002 30636 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 18:35:06.796082 30642 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:35:06.796118 30641 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:35:06.796154 30644 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 18:35:06.796434 30636 server_base.cc:1034] running on GCE node
I20260806 18:35:06.796763 30636 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 18:35:06.797287 30636 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 18:35:06.798449 30636 hybrid_clock.cc:648] HybridClock initialized: now 1786041306798432 us; error 30 us; skew 500 ppm
I20260806 18:35:06.800158 30636 webserver.cc:460] Webserver started at http://127.19.194.3:38553/ using document root <none> and password file <none>
I20260806 18:35:06.800738 30636 fs_manager.cc:359] Metadata directory not provided
I20260806 18:35:06.800799 30636 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 18:35:06.801046 30636 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 18:35:06.802923 30636 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786041246094603-20232-0/minicluster-data/ts-2/data/instance:
uuid: "f9173e1d18934b589fcdca24750dba33"
format_stamp: "Formatted at 2026-08-06 18:35:06 on dist-test-slave-1xrh"
server_key: "4a53b77954b47f8781fac72d11114048"
server_key_iv: "15086b3e3a2fe91e2ceffaffa70af634"
server_key_version: "encryptionkey@0"
I20260806 18:35:06.803521 30636 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786041246094603-20232-0/minicluster-data/ts-2/wal/instance:
uuid: "f9173e1d18934b589fcdca24750dba33"
format_stamp: "Formatted at 2026-08-06 18:35:06 on dist-test-slave-1xrh"
server_key: "4a53b77954b47f8781fac72d11114048"
server_key_iv: "15086b3e3a2fe91e2ceffaffa70af634"
server_key_version: "encryptionkey@0"
I20260806 18:35:06.806869 30636 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.000s
I20260806 18:35:06.809227 30650 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 18:35:06.810390 30636 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.000s sys 0.002s
I20260806 18:35:06.810516 30636 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786041246094603-20232-0/minicluster-data/ts-2/data,/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786041246094603-20232-0/minicluster-data/ts-2/wal
uuid: "f9173e1d18934b589fcdca24750dba33"
format_stamp: "Formatted at 2026-08-06 18:35:06 on dist-test-slave-1xrh"
server_key: "4a53b77954b47f8781fac72d11114048"
server_key_iv: "15086b3e3a2fe91e2ceffaffa70af634"
server_key_version: "encryptionkey@0"
I20260806 18:35:06.810622 30636 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786041246094603-20232-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786041246094603-20232-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786041246094603-20232-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 18:35:06.815165 30636 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 18:35:06.815750 30636 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 18:35:06.815928 30636 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 18:35:06.816442 30636 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 18:35:06.817322 30636 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 18:35:06.817392 30636 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 18:35:06.817461 30636 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 18:35:06.817514 30636 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 18:35:06.826161 30636 rpc_server.cc:307] RPC server started. Bound to: 127.19.194.3:44269
I20260806 18:35:06.826197 30768 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.194.3:44269 every 8 connection(s)
I20260806 18:35:06.826990 30636 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786041246094603-20232-0/minicluster-data/ts-2/data/info.pb
I20260806 18:35:06.832850 30344 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:35:06.828770 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.194.3:33939 (local address 127.19.194.62:43071)
0806 18:35:06.828908 (+ 138us) server_negotiation.cc:206] Beginning negotiation
0806 18:35:06.828912 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 18:35:06.829843 (+ 931us) server_negotiation.cc:407] Connection header received
0806 18:35:06.831198 (+ 1355us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:35:06.831201 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:35:06.831250 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:35:06.831325 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:35:06.832365 (+ 1040us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:35:06.832369 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:35:06.832372 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 18:35:06.832384 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:35:06.832405 (+ 21us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 18:35:06.832421 (+ 16us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:35:06.832515 (+ 94us) server_negotiation.cc:1035] Waiting for connection context
0806 18:35:06.832667 (+ 152us) server_negotiation.cc:299] Negotiation successful
0806 18:35:06.832714 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260806 18:35:06.833599 30771 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:35:06.829081 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.194.62:43071 (local address 127.19.194.3:33939)
0806 18:35:06.829698 (+ 617us) negotiation.cc:107] Waiting for socket to connect
0806 18:35:06.829737 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0806 18:35:06.830952 (+ 1215us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:35:06.831479 (+ 527us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:35:06.831486 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:35:06.831573 (+ 87us) client_negotiation.cc:189] Negotiated authn=SASL
0806 18:35:06.832101 (+ 528us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0806 18:35:06.832103 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0806 18:35:06.832153 (+ 50us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0806 18:35:06.832216 (+ 63us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 18:35:06.832500 (+ 284us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 18:35:06.832510 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 18:35:06.832515 (+ 5us) client_negotiation.cc:769] Sending connection context
0806 18:35:06.832702 (+ 187us) client_negotiation.cc:240] Negotiation successful
0806 18:35:06.832920 (+ 218us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":241,"thread_start_us":103,"threads_started":1}
I20260806 18:35:06.834679 30769 heartbeater.cc:347] Connected to a master server at 127.19.194.62:43071
I20260806 18:35:06.834741 20232 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJW69tn/build/debug/bin/kudu as pid 30636
I20260806 18:35:06.834870 20232 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786041246094603-20232-0/minicluster-data/ts-2/wal/instance
I20260806 18:35:06.834942 30769 heartbeater.cc:464] Registering TS with master...
I20260806 18:35:06.835237 20232 external_mini_cluster.cc:1428] Setting key 60799d537e9e55adabd0ed073b3b6a62
I20260806 18:35:06.835414 30769 heartbeater.cc:511] Master 127.19.194.62:43071 requested a full tablet report, sending...
I20260806 18:35:06.836495 30304 ts_manager.cc:194] Registered new tserver with Master: f9173e1d18934b589fcdca24750dba33 (127.19.194.3:44269)
I20260806 18:35:06.837189 20232 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
W20260806 18:35:06.840014 30344 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 18:35:06.839089 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42138 (local address 127.19.194.62:43071)
0806 18:35:06.839224 (+ 135us) server_negotiation.cc:206] Beginning negotiation
0806 18:35:06.839228 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 18:35:06.839421 (+ 193us) server_negotiation.cc:407] Connection header received
0806 18:35:06.839550 (+ 129us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:35:06.839552 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:35:06.839578 (+ 26us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:35:06.839639 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:35:06.839897 (+ 258us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:42138: BlockingRecv error: recv got EOF from 127.0.0.1:42138 (error 108)
Metrics: {"server-negotiator.queue_time_us":52}
I20260806 18:35:06.840277 20232 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJW69tn/build/debug/bin/kudu with pid 30366
I20260806 18:35:06.849457 20232 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJW69tn/build/debug/bin/kudu with pid 30501
I20260806 18:35:06.855695 20232 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJW69tn/build/debug/bin/kudu with pid 30636
I20260806 18:35:06.861384 20232 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJW69tn/build/debug/bin/kudu with pid 30274
2026-08-06T18:35:06Z chronyd exiting
[ OK ] SecurityITest.TestRequireEncryptionInsecureCluster (790 ms)
[ RUN ] SecurityITest.TestRequireAuthenticationSecureCluster
Loading random data
Initializing database '/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786041246094603-20232-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 18:35:06 dist-test-slave-1xrh krb5kdc[30779](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 18:35:06 dist-test-slave-1xrh krb5kdc[30779](info): set up 2 sockets
Aug 06 18:35:06 dist-test-slave-1xrh krb5kdc[30779](info): commencing operation
krb5kdc: starting...
W20260806 18:35:07.948745 20232 mini_kdc.cc:121] Time spent starting KDC: real 1.065s user 0.005s sys 0.001s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786041246094603-20232-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786041246094603-20232-0/krb5kdc/test-user.keytab.
Aug 06 18:35:08 dist-test-slave-1xrh krb5kdc[30779](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041308, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T18:35:08Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T18:35:08Z Disabled control of system clock
WARNING: no policy specified for kudu/127.19.194.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.19.194.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.19.194.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786041246094603-20232-0/krb5kdc/kudu_127.19.194.62.keytab.
Entry for principal kudu/127.19.194.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786041246094603-20232-0/krb5kdc/kudu_127.19.194.62.keytab.
I20260806 18:35:08.070667 20232 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJW69tn/build/debug/bin/kudu
/tmp/dist-test-taskJW69tn/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786041246094603-20232-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786041246094603-20232-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786041246094603-20232-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786041246094603-20232-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.19.194.62:41429
--webserver_interface=127.19.194.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786041246094603-20232-0/krb5kdc/kudu_127.19.194.62.keytab
--principal=kudu/127.19.194.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.19.194.20:40647
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.19.194.62:41429
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786041246094603-20232-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786041246094603-20232-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786041246094603-20232-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 18:35:08.177582 30792 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 18:35:08.177870 30792 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 18:35:08.177956 30792 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 18:35:08.181708 30792 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 18:35:08.181782 30792 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 18:35:08.181820 30792 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 18:35:08.181877 30792 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 18:35:08.181936 30792 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 18:35:08.187080 30792 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.194.20:40647
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786041246094603-20232-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786041246094603-20232-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.19.194.62:41429
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786041246094603-20232-0/krb5kdc/kudu_127.19.194.62.keytab
--principal=kudu/127.19.194.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.19.194.62:41429
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786041246094603-20232-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.19.194.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30792
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786041246094603-20232-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 17:59:13 UTC on ccycloud.quasar-wvsnzx.root.comops.site
build id 4571028
I20260806 18:35:08.188059 30792 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 18:35:08.189002 30792 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 18:35:08.197217 30797 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 18:35:08.197326 30792 server_base.cc:1034] running on GCE node
W20260806 18:35:08.197222 30798 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:35:08.197227 30800 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 18:35:08.197954 30792 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 18:35:08.198998 30792 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 18:35:08.200189 30792 hybrid_clock.cc:648] HybridClock initialized: now 1786041308200170 us; error 28 us; skew 500 ppm
Aug 06 18:35:08 dist-test-slave-1xrh krb5kdc[30779](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041308, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.194.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 18:35:08.202809 30792 init.cc:377] Logged in from keytab as kudu/127.19.194.62@KRBTEST.COM (short username kudu)
I20260806 18:35:08.203732 30792 webserver.cc:460] Webserver started at http://127.19.194.62:39749/ using document root <none> and password file <none>
I20260806 18:35:08.204291 30792 fs_manager.cc:359] Metadata directory not provided
I20260806 18:35:08.204382 30792 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 18:35:08.204625 30792 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 18:35:08.206602 30792 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786041246094603-20232-0/minicluster-data/master-0/data/instance:
uuid: "040776b2955740a28506c9a36d2b6424"
format_stamp: "Formatted at 2026-08-06 18:35:08 on dist-test-slave-1xrh"
server_key: "9b2f9ba5fd37f839648c7becd159cebc"
server_key_iv: "79d59202a3c85dc4bb39ea71137db724"
server_key_version: "encryptionkey@0"
I20260806 18:35:08.207226 30792 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786041246094603-20232-0/minicluster-data/master-0/wal/instance:
uuid: "040776b2955740a28506c9a36d2b6424"
format_stamp: "Formatted at 2026-08-06 18:35:08 on dist-test-slave-1xrh"
server_key: "9b2f9ba5fd37f839648c7becd159cebc"
server_key_iv: "79d59202a3c85dc4bb39ea71137db724"
server_key_version: "encryptionkey@0"
I20260806 18:35:08.210696 30792 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.000s
I20260806 18:35:08.212983 30807 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 18:35:08.214125 30792 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260806 18:35:08.214270 30792 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786041246094603-20232-0/minicluster-data/master-0/data,/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786041246094603-20232-0/minicluster-data/master-0/wal
uuid: "040776b2955740a28506c9a36d2b6424"
format_stamp: "Formatted at 2026-08-06 18:35:08 on dist-test-slave-1xrh"
server_key: "9b2f9ba5fd37f839648c7becd159cebc"
server_key_iv: "79d59202a3c85dc4bb39ea71137db724"
server_key_version: "encryptionkey@0"
I20260806 18:35:08.214375 30792 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786041246094603-20232-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786041246094603-20232-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786041246094603-20232-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 18:35:08.231849 30792 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 18:35:08.234414 30792 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 18:35:08.234617 30792 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 18:35:08.241497 30792 rpc_server.cc:307] RPC server started. Bound to: 127.19.194.62:41429
I20260806 18:35:08.241762 30862 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.194.62:41429 every 8 connection(s)
I20260806 18:35:08.242419 30792 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786041246094603-20232-0/minicluster-data/master-0/data/info.pb
I20260806 18:35:08.245206 30863 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 18:35:08.246426 20232 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJW69tn/build/debug/bin/kudu as pid 30792
I20260806 18:35:08.246553 20232 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786041246094603-20232-0/minicluster-data/master-0/wal/instance
I20260806 18:35:08.246878 20232 external_mini_cluster.cc:1428] Setting key b105b18fd71dd2134ea651c6fb73e496
I20260806 18:35:08.251550 30863 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 040776b2955740a28506c9a36d2b6424: Bootstrap starting.
I20260806 18:35:08.254153 30863 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 040776b2955740a28506c9a36d2b6424: Neither blocks nor log segments found. Creating new log.
I20260806 18:35:08.255160 30863 log.cc:824] T 00000000000000000000000000000000 P 040776b2955740a28506c9a36d2b6424: Log is configured to *not* fsync() on all Append() calls
Aug 06 18:35:08 dist-test-slave-1xrh krb5kdc[30779](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041308, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.19.194.62@KRBTEST.COM
I20260806 18:35:08.257109 30863 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 040776b2955740a28506c9a36d2b6424: No bootstrap required, opened a new log
I20260806 18:35:08.261072 30866 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:35:08.248261 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51276 (local address 127.19.194.62:41429)
0806 18:35:08.248642 (+ 381us) server_negotiation.cc:206] Beginning negotiation
0806 18:35:08.248659 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0806 18:35:08.248692 (+ 33us) server_negotiation.cc:407] Connection header received
0806 18:35:08.249316 (+ 624us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:35:08.249331 (+ 15us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:35:08.249647 (+ 316us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:35:08.249902 (+ 255us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:35:08.250817 (+ 915us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:35:08.251866 (+ 1049us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:35:08.253042 (+ 1176us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:35:08.253487 (+ 445us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:35:08.253598 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:35:08.256502 (+ 2904us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:35:08.256507 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:35:08.256518 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 18:35:08.256542 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:35:08.258233 (+ 1691us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:35:08.258728 (+ 495us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:35:08.258733 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:35:08.258737 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:35:08.258840 (+ 103us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:35:08.259229 (+ 389us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:35:08.259231 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:35:08.259232 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:35:08.259638 (+ 406us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:35:08.259778 (+ 140us) server_negotiation.cc:1035] Waiting for connection context
0806 18:35:08.260077 (+ 299us) server_negotiation.cc:299] Negotiation successful
0806 18:35:08.260366 (+ 289us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":246,"thread_start_us":129,"threads_started":1}
I20260806 18:35:08.261078 30863 raft_consensus.cc:348] T 00000000000000000000000000000000 P 040776b2955740a28506c9a36d2b6424 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "040776b2955740a28506c9a36d2b6424" member_type: VOTER last_known_addr { host: "127.19.194.62" port: 41429 } }
I20260806 18:35:08.261447 30863 raft_consensus.cc:374] T 00000000000000000000000000000000 P 040776b2955740a28506c9a36d2b6424 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 18:35:08.261523 30863 raft_consensus.cc:729] T 00000000000000000000000000000000 P 040776b2955740a28506c9a36d2b6424 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 040776b2955740a28506c9a36d2b6424, State: Initialized, Role: FOLLOWER
I20260806 18:35:08.261920 30863 consensus_queue.cc:260] T 00000000000000000000000000000000 P 040776b2955740a28506c9a36d2b6424 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "040776b2955740a28506c9a36d2b6424" member_type: VOTER last_known_addr { host: "127.19.194.62" port: 41429 } }
I20260806 18:35:08.262048 30863 raft_consensus.cc:388] T 00000000000000000000000000000000 P 040776b2955740a28506c9a36d2b6424 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 18:35:08.262157 30863 raft_consensus.cc:482] T 00000000000000000000000000000000 P 040776b2955740a28506c9a36d2b6424 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 18:35:08.262259 30863 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 040776b2955740a28506c9a36d2b6424 [term 0 FOLLOWER]: Advancing to term 1
I20260806 18:35:08.263108 30863 raft_consensus.cc:504] T 00000000000000000000000000000000 P 040776b2955740a28506c9a36d2b6424 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "040776b2955740a28506c9a36d2b6424" member_type: VOTER last_known_addr { host: "127.19.194.62" port: 41429 } }
I20260806 18:35:08.263489 30863 leader_election.cc:302] T 00000000000000000000000000000000 P 040776b2955740a28506c9a36d2b6424 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 040776b2955740a28506c9a36d2b6424; no voters:
I20260806 18:35:08.263803 30863 leader_election.cc:288] T 00000000000000000000000000000000 P 040776b2955740a28506c9a36d2b6424 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 18:35:08.263907 30868 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 040776b2955740a28506c9a36d2b6424 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 18:35:08.264163 30868 raft_consensus.cc:686] T 00000000000000000000000000000000 P 040776b2955740a28506c9a36d2b6424 [term 1 LEADER]: Becoming Leader. State: Replica: 040776b2955740a28506c9a36d2b6424, State: Running, Role: LEADER
I20260806 18:35:08.264504 30868 consensus_queue.cc:237] T 00000000000000000000000000000000 P 040776b2955740a28506c9a36d2b6424 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "040776b2955740a28506c9a36d2b6424" member_type: VOTER last_known_addr { host: "127.19.194.62" port: 41429 } }
I20260806 18:35:08.264983 30863 sys_catalog.cc:564] T 00000000000000000000000000000000 P 040776b2955740a28506c9a36d2b6424 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 18:35:08.266060 30869 sys_catalog.cc:455] T 00000000000000000000000000000000 P 040776b2955740a28506c9a36d2b6424 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "040776b2955740a28506c9a36d2b6424" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "040776b2955740a28506c9a36d2b6424" member_type: VOTER last_known_addr { host: "127.19.194.62" port: 41429 } } }
I20260806 18:35:08.266330 30870 sys_catalog.cc:455] T 00000000000000000000000000000000 P 040776b2955740a28506c9a36d2b6424 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 040776b2955740a28506c9a36d2b6424. Latest consensus state: current_term: 1 leader_uuid: "040776b2955740a28506c9a36d2b6424" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "040776b2955740a28506c9a36d2b6424" member_type: VOTER last_known_addr { host: "127.19.194.62" port: 41429 } } }
I20260806 18:35:08.266506 30870 sys_catalog.cc:458] T 00000000000000000000000000000000 P 040776b2955740a28506c9a36d2b6424 [sys.catalog]: This master's current role is: LEADER
I20260806 18:35:08.266448 30869 sys_catalog.cc:458] T 00000000000000000000000000000000 P 040776b2955740a28506c9a36d2b6424 [sys.catalog]: This master's current role is: LEADER
I20260806 18:35:08.266920 30878 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 18:35:08.269610 30878 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 18:35:08.274477 30878 catalog_manager.cc:1347] Generated new cluster ID: fabaea64b64c4d33bff0a725690b79da
I20260806 18:35:08.274549 30878 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 18:35:08.284265 30878 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 18:35:08.285352 30878 catalog_manager.cc:1504] Loading token signing keys...
I20260806 18:35:08.293718 30878 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 040776b2955740a28506c9a36d2b6424: Generated new TSK 0
I20260806 18:35:08.294371 30878 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.19.194.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.19.194.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.19.194.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786041246094603-20232-0/krb5kdc/kudu_127.19.194.1.keytab.
Entry for principal kudu/127.19.194.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786041246094603-20232-0/krb5kdc/kudu_127.19.194.1.keytab.
I20260806 18:35:08.327507 20232 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJW69tn/build/debug/bin/kudu
/tmp/dist-test-taskJW69tn/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786041246094603-20232-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786041246094603-20232-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786041246094603-20232-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786041246094603-20232-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.19.194.1:0
--local_ip_for_outbound_sockets=127.19.194.1
--webserver_interface=127.19.194.1
--webserver_port=0
--tserver_master_addrs=127.19.194.62:41429
--keytab_file=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786041246094603-20232-0/krb5kdc/kudu_127.19.194.1.keytab
--principal=kudu/127.19.194.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.19.194.20:40647
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786041246094603-20232-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786041246094603-20232-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786041246094603-20232-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 18:35:08.438596 30889 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 18:35:08.438877 30889 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 18:35:08.438958 30889 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 18:35:08.442937 30889 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 18:35:08.443017 30889 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 18:35:08.443140 30889 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.19.194.1
I20260806 18:35:08.448263 30889 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.194.20:40647
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786041246094603-20232-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786041246094603-20232-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786041246094603-20232-0/krb5kdc/kudu_127.19.194.1.keytab
--principal=kudu/127.19.194.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.19.194.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786041246094603-20232-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.19.194.1
--webserver_port=0
--tserver_master_addrs=127.19.194.62:41429
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30889
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.19.194.1
--log_dir=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786041246094603-20232-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 17:59:13 UTC on ccycloud.quasar-wvsnzx.root.comops.site
build id 4571028
I20260806 18:35:08.449239 30889 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 18:35:08.450177 30889 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 18:35:08.459419 30897 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:35:08.459352 30894 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:35:08.459450 30895 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 18:35:08.459460 30889 server_base.cc:1034] running on GCE node
I20260806 18:35:08.460005 30889 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 18:35:08.460601 30889 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 18:35:08.461790 30889 hybrid_clock.cc:648] HybridClock initialized: now 1786041308461766 us; error 35 us; skew 500 ppm
Aug 06 18:35:08 dist-test-slave-1xrh krb5kdc[30779](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041308, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.194.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 18:35:08.464553 30889 init.cc:377] Logged in from keytab as kudu/127.19.194.1@KRBTEST.COM (short username kudu)
I20260806 18:35:08.465538 30889 webserver.cc:460] Webserver started at http://127.19.194.1:36211/ using document root <none> and password file <none>
I20260806 18:35:08.466079 30889 fs_manager.cc:359] Metadata directory not provided
I20260806 18:35:08.466204 30889 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 18:35:08.466439 30889 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 18:35:08.468503 30889 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786041246094603-20232-0/minicluster-data/ts-0/data/instance:
uuid: "ec51f83c60694c8eb32b61ddd2e9b936"
format_stamp: "Formatted at 2026-08-06 18:35:08 on dist-test-slave-1xrh"
server_key: "b124b230d84f8320eef961a9729ac46e"
server_key_iv: "da83f7ecc2009ae6f7fe4301e16b7e87"
server_key_version: "encryptionkey@0"
I20260806 18:35:08.469002 30889 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786041246094603-20232-0/minicluster-data/ts-0/wal/instance:
uuid: "ec51f83c60694c8eb32b61ddd2e9b936"
format_stamp: "Formatted at 2026-08-06 18:35:08 on dist-test-slave-1xrh"
server_key: "b124b230d84f8320eef961a9729ac46e"
server_key_iv: "da83f7ecc2009ae6f7fe4301e16b7e87"
server_key_version: "encryptionkey@0"
I20260806 18:35:08.472558 30889 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20260806 18:35:08.474709 30904 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 18:35:08.475924 30889 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260806 18:35:08.476015 30889 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786041246094603-20232-0/minicluster-data/ts-0/data,/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786041246094603-20232-0/minicluster-data/ts-0/wal
uuid: "ec51f83c60694c8eb32b61ddd2e9b936"
format_stamp: "Formatted at 2026-08-06 18:35:08 on dist-test-slave-1xrh"
server_key: "b124b230d84f8320eef961a9729ac46e"
server_key_iv: "da83f7ecc2009ae6f7fe4301e16b7e87"
server_key_version: "encryptionkey@0"
I20260806 18:35:08.476090 30889 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786041246094603-20232-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786041246094603-20232-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786041246094603-20232-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 18:35:08.495033 30889 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 18:35:08.497866 30889 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 18:35:08.498013 30889 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 18:35:08.498502 30889 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 18:35:08.499313 30889 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 18:35:08.499370 30889 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 18:35:08.499410 30889 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 18:35:08.499432 30889 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 18:35:08.508846 30889 rpc_server.cc:307] RPC server started. Bound to: 127.19.194.1:37047
I20260806 18:35:08.508885 31020 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.194.1:37047 every 8 connection(s)
I20260806 18:35:08.509735 30889 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786041246094603-20232-0/minicluster-data/ts-0/data/info.pb
I20260806 18:35:08.514426 20232 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJW69tn/build/debug/bin/kudu as pid 30889
I20260806 18:35:08.514554 20232 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786041246094603-20232-0/minicluster-data/ts-0/wal/instance
I20260806 18:35:08.514854 20232 external_mini_cluster.cc:1428] Setting key 9b0e981af265a90ac4d34b8358b0ee44
Aug 06 18:35:08 dist-test-slave-1xrh krb5kdc[30779](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041308, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.194.1@KRBTEST.COM for kudu/127.19.194.62@KRBTEST.COM
I20260806 18:35:08.523875 30866 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:35:08.511514 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.194.1:40991 (local address 127.19.194.62:41429)
0806 18:35:08.511673 (+ 159us) server_negotiation.cc:206] Beginning negotiation
0806 18:35:08.511678 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 18:35:08.512491 (+ 813us) server_negotiation.cc:407] Connection header received
0806 18:35:08.513837 (+ 1346us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:35:08.513839 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:35:08.513885 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:35:08.513975 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:35:08.515376 (+ 1401us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:35:08.516080 (+ 704us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:35:08.517232 (+ 1152us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:35:08.517488 (+ 256us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:35:08.517597 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:35:08.520170 (+ 2573us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:35:08.520173 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:35:08.520176 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 18:35:08.520201 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:35:08.521702 (+ 1501us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:35:08.522268 (+ 566us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:35:08.522276 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:35:08.522278 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:35:08.522356 (+ 78us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:35:08.522704 (+ 348us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:35:08.522708 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:35:08.522710 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:35:08.522897 (+ 187us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:35:08.523008 (+ 111us) server_negotiation.cc:1035] Waiting for connection context
0806 18:35:08.523557 (+ 549us) server_negotiation.cc:299] Negotiation successful
0806 18:35:08.523677 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
WARNING: no policy specified for kudu/127.19.194.2@KRBTEST.COM; defaulting to no policy
I20260806 18:35:08.524647 31024 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:35:08.511910 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.194.62:41429 (local address 127.19.194.1:40991)
0806 18:35:08.512348 (+ 438us) negotiation.cc:107] Waiting for socket to connect
0806 18:35:08.512388 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0806 18:35:08.513592 (+ 1204us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:35:08.514175 (+ 583us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:35:08.514184 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:35:08.514606 (+ 422us) client_negotiation.cc:189] Negotiated authn=SASL
0806 18:35:08.515237 (+ 631us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:35:08.515253 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:35:08.516220 (+ 967us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:35:08.516223 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:35:08.517100 (+ 877us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:35:08.517113 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:35:08.517673 (+ 560us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:35:08.517678 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:35:08.517838 (+ 160us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:35:08.518505 (+ 667us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 18:35:08.518521 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0806 18:35:08.519991 (+ 1470us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 18:35:08.521837 (+ 1846us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:35:08.521842 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:35:08.521856 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:35:08.522146 (+ 290us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:35:08.522453 (+ 307us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:35:08.522456 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:35:08.522458 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:35:08.522604 (+ 146us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:35:08.523008 (+ 404us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 18:35:08.523014 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 18:35:08.523308 (+ 294us) client_negotiation.cc:769] Sending connection context
0806 18:35:08.523511 (+ 203us) client_negotiation.cc:240] Negotiation successful
0806 18:35:08.523827 (+ 316us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":259,"thread_start_us":109,"threads_started":1}
I20260806 18:35:08.525911 31021 heartbeater.cc:347] Connected to a master server at 127.19.194.62:41429
I20260806 18:35:08.526254 31021 heartbeater.cc:464] Registering TS with master...
I20260806 18:35:08.526799 31021 heartbeater.cc:511] Master 127.19.194.62:41429 requested a full tablet report, sending...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.19.194.2@KRBTEST.COM" created.
I20260806 18:35:08.528383 30825 ts_manager.cc:194] Registered new tserver with Master: ec51f83c60694c8eb32b61ddd2e9b936 (127.19.194.1:37047)
I20260806 18:35:08.529588 30825 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.19.194.1@KRBTEST.COM'} at 127.19.194.1:40991
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.19.194.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786041246094603-20232-0/krb5kdc/kudu_127.19.194.2.keytab.
Entry for principal kudu/127.19.194.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786041246094603-20232-0/krb5kdc/kudu_127.19.194.2.keytab.
I20260806 18:35:08.543609 20232 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJW69tn/build/debug/bin/kudu
/tmp/dist-test-taskJW69tn/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786041246094603-20232-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786041246094603-20232-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786041246094603-20232-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786041246094603-20232-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.19.194.2:0
--local_ip_for_outbound_sockets=127.19.194.2
--webserver_interface=127.19.194.2
--webserver_port=0
--tserver_master_addrs=127.19.194.62:41429
--keytab_file=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786041246094603-20232-0/krb5kdc/kudu_127.19.194.2.keytab
--principal=kudu/127.19.194.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.19.194.20:40647
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786041246094603-20232-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786041246094603-20232-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786041246094603-20232-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 18:35:08.653787 31027 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 18:35:08.654014 31027 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 18:35:08.654055 31027 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 18:35:08.657909 31027 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 18:35:08.657956 31027 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 18:35:08.658010 31027 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.19.194.2
I20260806 18:35:08.662902 31027 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.194.20:40647
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786041246094603-20232-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786041246094603-20232-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786041246094603-20232-0/krb5kdc/kudu_127.19.194.2.keytab
--principal=kudu/127.19.194.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.19.194.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786041246094603-20232-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.19.194.2
--webserver_port=0
--tserver_master_addrs=127.19.194.62:41429
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31027
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.19.194.2
--log_dir=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786041246094603-20232-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 17:59:13 UTC on ccycloud.quasar-wvsnzx.root.comops.site
build id 4571028
I20260806 18:35:08.663892 31027 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 18:35:08.664938 31027 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 18:35:08.674460 31035 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:35:08.674504 31032 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 18:35:08.674572 31027 server_base.cc:1034] running on GCE node
W20260806 18:35:08.674468 31033 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 18:35:08.675020 31027 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 18:35:08.675716 31027 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 18:35:08.676875 31027 hybrid_clock.cc:648] HybridClock initialized: now 1786041308676863 us; error 24 us; skew 500 ppm
Aug 06 18:35:08 dist-test-slave-1xrh krb5kdc[30779](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041308, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.194.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 18:35:08.679854 31027 init.cc:377] Logged in from keytab as kudu/127.19.194.2@KRBTEST.COM (short username kudu)
I20260806 18:35:08.680671 31027 webserver.cc:460] Webserver started at http://127.19.194.2:38423/ using document root <none> and password file <none>
I20260806 18:35:08.681213 31027 fs_manager.cc:359] Metadata directory not provided
I20260806 18:35:08.681304 31027 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 18:35:08.681537 31027 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 18:35:08.683561 31027 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786041246094603-20232-0/minicluster-data/ts-1/data/instance:
uuid: "4d61ee914f7349a08f7894a0f8122234"
format_stamp: "Formatted at 2026-08-06 18:35:08 on dist-test-slave-1xrh"
server_key: "4a4d4525b254bfda838e021c68bf13fe"
server_key_iv: "2bee640b598bfef2fb390a852b86bd2c"
server_key_version: "encryptionkey@0"
I20260806 18:35:08.684126 31027 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786041246094603-20232-0/minicluster-data/ts-1/wal/instance:
uuid: "4d61ee914f7349a08f7894a0f8122234"
format_stamp: "Formatted at 2026-08-06 18:35:08 on dist-test-slave-1xrh"
server_key: "4a4d4525b254bfda838e021c68bf13fe"
server_key_iv: "2bee640b598bfef2fb390a852b86bd2c"
server_key_version: "encryptionkey@0"
I20260806 18:35:08.687630 31027 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.000s
I20260806 18:35:08.690073 31042 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 18:35:08.691250 31027 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260806 18:35:08.691387 31027 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786041246094603-20232-0/minicluster-data/ts-1/data,/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786041246094603-20232-0/minicluster-data/ts-1/wal
uuid: "4d61ee914f7349a08f7894a0f8122234"
format_stamp: "Formatted at 2026-08-06 18:35:08 on dist-test-slave-1xrh"
server_key: "4a4d4525b254bfda838e021c68bf13fe"
server_key_iv: "2bee640b598bfef2fb390a852b86bd2c"
server_key_version: "encryptionkey@0"
I20260806 18:35:08.691493 31027 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786041246094603-20232-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786041246094603-20232-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786041246094603-20232-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 18:35:08.708258 31027 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 18:35:08.711421 31027 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 18:35:08.711616 31027 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 18:35:08.712155 31027 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 18:35:08.712939 31027 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 18:35:08.713011 31027 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 18:35:08.713071 31027 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 18:35:08.713114 31027 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 18:35:08.721688 31027 rpc_server.cc:307] RPC server started. Bound to: 127.19.194.2:38277
I20260806 18:35:08.721755 31160 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.194.2:38277 every 8 connection(s)
I20260806 18:35:08.722553 31027 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786041246094603-20232-0/minicluster-data/ts-1/data/info.pb
I20260806 18:35:08.729238 20232 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJW69tn/build/debug/bin/kudu as pid 31027
I20260806 18:35:08.729357 20232 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786041246094603-20232-0/minicluster-data/ts-1/wal/instance
I20260806 18:35:08.729655 20232 external_mini_cluster.cc:1428] Setting key 60676f0f987e95f0a9a42836429539d4
Aug 06 18:35:08 dist-test-slave-1xrh krb5kdc[30779](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041308, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.194.2@KRBTEST.COM for kudu/127.19.194.62@KRBTEST.COM
I20260806 18:35:08.736500 30866 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:35:08.724306 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.194.2:55285 (local address 127.19.194.62:41429)
0806 18:35:08.724452 (+ 146us) server_negotiation.cc:206] Beginning negotiation
0806 18:35:08.724456 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 18:35:08.725233 (+ 777us) server_negotiation.cc:407] Connection header received
0806 18:35:08.726601 (+ 1368us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:35:08.726604 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:35:08.726669 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:35:08.726783 (+ 114us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:35:08.727987 (+ 1204us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:35:08.728509 (+ 522us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:35:08.730010 (+ 1501us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:35:08.730348 (+ 338us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:35:08.730467 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:35:08.733055 (+ 2588us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:35:08.733060 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:35:08.733065 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 18:35:08.733095 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:35:08.734484 (+ 1389us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:35:08.734993 (+ 509us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:35:08.735002 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:35:08.735004 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:35:08.735081 (+ 77us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:35:08.735454 (+ 373us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:35:08.735458 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:35:08.735460 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:35:08.735644 (+ 184us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:35:08.735751 (+ 107us) server_negotiation.cc:1035] Waiting for connection context
0806 18:35:08.736214 (+ 463us) server_negotiation.cc:299] Negotiation successful
0806 18:35:08.736332 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260806 18:35:08.737411 31163 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:35:08.724653 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.194.62:41429 (local address 127.19.194.2:55285)
0806 18:35:08.725093 (+ 440us) negotiation.cc:107] Waiting for socket to connect
0806 18:35:08.725133 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0806 18:35:08.726377 (+ 1244us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:35:08.726952 (+ 575us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:35:08.726958 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:35:08.727368 (+ 410us) client_negotiation.cc:189] Negotiated authn=SASL
0806 18:35:08.727821 (+ 453us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:35:08.727833 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:35:08.728641 (+ 808us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:35:08.728647 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:35:08.729887 (+ 1240us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:35:08.729898 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:35:08.730503 (+ 605us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:35:08.730506 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:35:08.730638 (+ 132us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:35:08.731266 (+ 628us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 18:35:08.731286 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0806 18:35:08.732887 (+ 1601us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 18:35:08.734616 (+ 1729us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:35:08.734622 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:35:08.734635 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:35:08.734882 (+ 247us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:35:08.735209 (+ 327us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:35:08.735212 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:35:08.735214 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:35:08.735353 (+ 139us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:35:08.735752 (+ 399us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 18:35:08.735759 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 18:35:08.735981 (+ 222us) client_negotiation.cc:769] Sending connection context
0806 18:35:08.736168 (+ 187us) client_negotiation.cc:240] Negotiation successful
0806 18:35:08.736462 (+ 294us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":266,"thread_start_us":116,"threads_started":1}
I20260806 18:35:08.738592 31161 heartbeater.cc:347] Connected to a master server at 127.19.194.62:41429
I20260806 18:35:08.738850 31161 heartbeater.cc:464] Registering TS with master...
WARNING: no policy specified for kudu/127.19.194.3@KRBTEST.COM; defaulting to no policy
I20260806 18:35:08.739456 31161 heartbeater.cc:511] Master 127.19.194.62:41429 requested a full tablet report, sending...
I20260806 18:35:08.740505 30825 ts_manager.cc:194] Registered new tserver with Master: 4d61ee914f7349a08f7894a0f8122234 (127.19.194.2:38277)
I20260806 18:35:08.741139 30825 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.19.194.2@KRBTEST.COM'} at 127.19.194.2:55285
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.19.194.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.19.194.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786041246094603-20232-0/krb5kdc/kudu_127.19.194.3.keytab.
Entry for principal kudu/127.19.194.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786041246094603-20232-0/krb5kdc/kudu_127.19.194.3.keytab.
I20260806 18:35:08.757561 20232 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJW69tn/build/debug/bin/kudu
/tmp/dist-test-taskJW69tn/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786041246094603-20232-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786041246094603-20232-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786041246094603-20232-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786041246094603-20232-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.19.194.3:0
--local_ip_for_outbound_sockets=127.19.194.3
--webserver_interface=127.19.194.3
--webserver_port=0
--tserver_master_addrs=127.19.194.62:41429
--keytab_file=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786041246094603-20232-0/krb5kdc/kudu_127.19.194.3.keytab
--principal=kudu/127.19.194.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.19.194.20:40647
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786041246094603-20232-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786041246094603-20232-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786041246094603-20232-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 18:35:08.865720 31166 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 18:35:08.865976 31166 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 18:35:08.866063 31166 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 18:35:08.870277 31166 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 18:35:08.870329 31166 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 18:35:08.870433 31166 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.19.194.3
I20260806 18:35:08.875304 31166 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.194.20:40647
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786041246094603-20232-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786041246094603-20232-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786041246094603-20232-0/krb5kdc/kudu_127.19.194.3.keytab
--principal=kudu/127.19.194.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.19.194.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786041246094603-20232-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.19.194.3
--webserver_port=0
--tserver_master_addrs=127.19.194.62:41429
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31166
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.19.194.3
--log_dir=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786041246094603-20232-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 17:59:13 UTC on ccycloud.quasar-wvsnzx.root.comops.site
build id 4571028
I20260806 18:35:08.876237 31166 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 18:35:08.877167 31166 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260806 18:35:08.886729 31166 server_base.cc:1034] running on GCE node
W20260806 18:35:08.886754 31174 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:35:08.886704 31171 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:35:08.886857 31172 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 18:35:08.887346 31166 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 18:35:08.887938 31166 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 18:35:08.889101 31166 hybrid_clock.cc:648] HybridClock initialized: now 1786041308889082 us; error 31 us; skew 500 ppm
Aug 06 18:35:08 dist-test-slave-1xrh krb5kdc[30779](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041308, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.194.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 18:35:08.891755 31166 init.cc:377] Logged in from keytab as kudu/127.19.194.3@KRBTEST.COM (short username kudu)
I20260806 18:35:08.892628 31166 webserver.cc:460] Webserver started at http://127.19.194.3:35153/ using document root <none> and password file <none>
I20260806 18:35:08.893172 31166 fs_manager.cc:359] Metadata directory not provided
I20260806 18:35:08.893231 31166 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 18:35:08.893389 31166 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 18:35:08.895361 31166 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786041246094603-20232-0/minicluster-data/ts-2/data/instance:
uuid: "f0dbb4cd1c4f48dc89152e03214c51de"
format_stamp: "Formatted at 2026-08-06 18:35:08 on dist-test-slave-1xrh"
server_key: "bbd4c53cd0f21d09a845bd9594643f09"
server_key_iv: "e6eca567dcd0b1c445c1ab238e5e61b1"
server_key_version: "encryptionkey@0"
I20260806 18:35:08.895870 31166 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786041246094603-20232-0/minicluster-data/ts-2/wal/instance:
uuid: "f0dbb4cd1c4f48dc89152e03214c51de"
format_stamp: "Formatted at 2026-08-06 18:35:08 on dist-test-slave-1xrh"
server_key: "bbd4c53cd0f21d09a845bd9594643f09"
server_key_iv: "e6eca567dcd0b1c445c1ab238e5e61b1"
server_key_version: "encryptionkey@0"
I20260806 18:35:08.899407 31166 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.000s
I20260806 18:35:08.901638 31181 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 18:35:08.902845 31166 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260806 18:35:08.902952 31166 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786041246094603-20232-0/minicluster-data/ts-2/data,/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786041246094603-20232-0/minicluster-data/ts-2/wal
uuid: "f0dbb4cd1c4f48dc89152e03214c51de"
format_stamp: "Formatted at 2026-08-06 18:35:08 on dist-test-slave-1xrh"
server_key: "bbd4c53cd0f21d09a845bd9594643f09"
server_key_iv: "e6eca567dcd0b1c445c1ab238e5e61b1"
server_key_version: "encryptionkey@0"
I20260806 18:35:08.903035 31166 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786041246094603-20232-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786041246094603-20232-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786041246094603-20232-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 18:35:08.924244 31166 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 18:35:08.926864 31166 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 18:35:08.927063 31166 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 18:35:08.927636 31166 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 18:35:08.928416 31166 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 18:35:08.928505 31166 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 18:35:08.928597 31166 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 18:35:08.928650 31166 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 18:35:08.937494 31166 rpc_server.cc:307] RPC server started. Bound to: 127.19.194.3:42945
I20260806 18:35:08.937527 31300 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.194.3:42945 every 8 connection(s)
I20260806 18:35:08.938325 31166 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786041246094603-20232-0/minicluster-data/ts-2/data/info.pb
I20260806 18:35:08.943608 20232 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJW69tn/build/debug/bin/kudu as pid 31166
I20260806 18:35:08.943737 20232 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786041246094603-20232-0/minicluster-data/ts-2/wal/instance
I20260806 18:35:08.944104 20232 external_mini_cluster.cc:1428] Setting key 91feef16fad83723826f97bfbe4e1523
Aug 06 18:35:08 dist-test-slave-1xrh krb5kdc[30779](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041308, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.194.3@KRBTEST.COM for kudu/127.19.194.62@KRBTEST.COM
I20260806 18:35:08.951758 30866 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:35:08.940109 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.194.3:49665 (local address 127.19.194.62:41429)
0806 18:35:08.940283 (+ 174us) server_negotiation.cc:206] Beginning negotiation
0806 18:35:08.940288 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 18:35:08.941104 (+ 816us) server_negotiation.cc:407] Connection header received
0806 18:35:08.942491 (+ 1387us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:35:08.942494 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:35:08.942566 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:35:08.942667 (+ 101us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:35:08.943909 (+ 1242us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:35:08.944601 (+ 692us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:35:08.945910 (+ 1309us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:35:08.946227 (+ 317us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:35:08.946320 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:35:08.948768 (+ 2448us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:35:08.948771 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:35:08.948773 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 18:35:08.948794 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:35:08.950003 (+ 1209us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:35:08.950448 (+ 445us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:35:08.950451 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:35:08.950452 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:35:08.950531 (+ 79us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:35:08.950819 (+ 288us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:35:08.950821 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:35:08.950822 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:35:08.950981 (+ 159us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:35:08.951056 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0806 18:35:08.951549 (+ 493us) server_negotiation.cc:299] Negotiation successful
0806 18:35:08.951641 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
I20260806 18:35:08.952507 31303 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:35:08.940492 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.194.62:41429 (local address 127.19.194.3:49665)
0806 18:35:08.940946 (+ 454us) negotiation.cc:107] Waiting for socket to connect
0806 18:35:08.940987 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0806 18:35:08.942268 (+ 1281us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:35:08.942841 (+ 573us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:35:08.942848 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:35:08.943251 (+ 403us) client_negotiation.cc:189] Negotiated authn=SASL
0806 18:35:08.943702 (+ 451us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:35:08.943747 (+ 45us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:35:08.944732 (+ 985us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:35:08.944737 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:35:08.945792 (+ 1055us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:35:08.945802 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:35:08.946370 (+ 568us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:35:08.946373 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:35:08.946508 (+ 135us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:35:08.947111 (+ 603us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 18:35:08.947178 (+ 67us) client_negotiation.cc:656] calling sasl_client_start()
0806 18:35:08.948625 (+ 1447us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 18:35:08.950107 (+ 1482us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:35:08.950111 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:35:08.950121 (+ 10us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:35:08.950363 (+ 242us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:35:08.950611 (+ 248us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:35:08.950614 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:35:08.950615 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:35:08.950741 (+ 126us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:35:08.951075 (+ 334us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 18:35:08.951079 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 18:35:08.951339 (+ 260us) client_negotiation.cc:769] Sending connection context
0806 18:35:08.951492 (+ 153us) client_negotiation.cc:240] Negotiation successful
0806 18:35:08.951720 (+ 228us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":277,"thread_start_us":119,"threads_started":1}
I20260806 18:35:08.953596 31301 heartbeater.cc:347] Connected to a master server at 127.19.194.62:41429
I20260806 18:35:08.953788 31301 heartbeater.cc:464] Registering TS with master...
I20260806 18:35:08.954190 31301 heartbeater.cc:511] Master 127.19.194.62:41429 requested a full tablet report, sending...
I20260806 18:35:08.955188 30825 ts_manager.cc:194] Registered new tserver with Master: f0dbb4cd1c4f48dc89152e03214c51de (127.19.194.3:42945)
I20260806 18:35:08.955753 30825 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.19.194.3@KRBTEST.COM'} at 127.19.194.3:49665
I20260806 18:35:08.957820 20232 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260806 18:35:08.966640 30866 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:35:08.959715 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51284 (local address 127.19.194.62:41429)
0806 18:35:08.959828 (+ 113us) server_negotiation.cc:206] Beginning negotiation
0806 18:35:08.959831 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 18:35:08.960013 (+ 182us) server_negotiation.cc:407] Connection header received
0806 18:35:08.960105 (+ 92us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:35:08.960107 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:35:08.960141 (+ 34us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:35:08.960204 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:35:08.960815 (+ 611us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:35:08.961281 (+ 466us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:35:08.962042 (+ 761us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:35:08.962239 (+ 197us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:35:08.962309 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:35:08.963675 (+ 1366us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:35:08.963678 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:35:08.963680 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 18:35:08.963701 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:35:08.965129 (+ 1428us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:35:08.965512 (+ 383us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:35:08.965515 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:35:08.965516 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:35:08.965610 (+ 94us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:35:08.965864 (+ 254us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:35:08.965867 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:35:08.965871 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:35:08.966076 (+ 205us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:35:08.966176 (+ 100us) server_negotiation.cc:1035] Waiting for connection context
0806 18:35:08.966435 (+ 259us) server_negotiation.cc:299] Negotiation successful
0806 18:35:08.966535 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260806 18:35:08.969305 30825 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:51284:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260806 18:35:08.972218 30825 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 18:35:08.986806 31312 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:35:08.980348 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.194.1:37047 (local address 127.0.0.1:60614)
0806 18:35:08.980662 (+ 314us) negotiation.cc:107] Waiting for socket to connect
0806 18:35:08.980706 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0806 18:35:08.980821 (+ 115us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:35:08.982226 (+ 1405us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:35:08.982232 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:35:08.982255 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 18:35:08.982406 (+ 151us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:35:08.982413 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:35:08.983628 (+ 1215us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:35:08.983635 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:35:08.984995 (+ 1360us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:35:08.985006 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:35:08.986127 (+ 1121us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:35:08.986134 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:35:08.986262 (+ 128us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:35:08.986362 (+ 100us) client_negotiation.cc:769] Sending connection context
0806 18:35:08.986469 (+ 107us) client_negotiation.cc:240] Negotiation successful
0806 18:35:08.986616 (+ 147us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":155,"mutex_wait_us":38,"thread_start_us":74,"threads_started":1}
I20260806 18:35:08.986824 31315 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:35:08.981000 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60614 (local address 127.19.194.1:37047)
0806 18:35:08.981665 (+ 665us) server_negotiation.cc:206] Beginning negotiation
0806 18:35:08.981682 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0806 18:35:08.981724 (+ 42us) server_negotiation.cc:407] Connection header received
0806 18:35:08.981807 (+ 83us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:35:08.981815 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:35:08.982062 (+ 247us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:35:08.982239 (+ 177us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 18:35:08.982549 (+ 310us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:35:08.983497 (+ 948us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:35:08.985140 (+ 1643us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:35:08.985985 (+ 845us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:35:08.986098 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:35:08.986245 (+ 147us) server_negotiation.cc:1035] Waiting for connection context
0806 18:35:08.986537 (+ 292us) server_negotiation.cc:299] Negotiation successful
0806 18:35:08.986655 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":534,"thread_start_us":121,"threads_started":1}
I20260806 18:35:08.986958 31313 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:35:08.980711 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.194.2:38277 (local address 127.0.0.1:58768)
0806 18:35:08.981297 (+ 586us) negotiation.cc:107] Waiting for socket to connect
0806 18:35:08.981310 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0806 18:35:08.981389 (+ 79us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:35:08.982440 (+ 1051us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:35:08.982443 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:35:08.982473 (+ 30us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 18:35:08.982580 (+ 107us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:35:08.982586 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:35:08.984238 (+ 1652us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:35:08.984242 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:35:08.985403 (+ 1161us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:35:08.985411 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:35:08.986223 (+ 812us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:35:08.986227 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:35:08.986328 (+ 101us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:35:08.986620 (+ 292us) client_negotiation.cc:769] Sending connection context
0806 18:35:08.986717 (+ 97us) client_negotiation.cc:240] Negotiation successful
0806 18:35:08.986776 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":521,"mutex_wait_us":338,"spinlock_wait_cycles":768,"thread_start_us":89,"threads_started":1}
I20260806 18:35:08.987496 31314 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:35:08.980456 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.194.3:42945 (local address 127.0.0.1:33148)
0806 18:35:08.981221 (+ 765us) negotiation.cc:107] Waiting for socket to connect
0806 18:35:08.981236 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0806 18:35:08.981335 (+ 99us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:35:08.982422 (+ 1087us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:35:08.982424 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:35:08.982443 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 18:35:08.982572 (+ 129us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:35:08.982578 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:35:08.984061 (+ 1483us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:35:08.984065 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:35:08.985435 (+ 1370us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:35:08.985446 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:35:08.987035 (+ 1589us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:35:08.987039 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:35:08.987159 (+ 120us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:35:08.987189 (+ 30us) client_negotiation.cc:769] Sending connection context
0806 18:35:08.987268 (+ 79us) client_negotiation.cc:240] Negotiation successful
0806 18:35:08.987344 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":693,"mutex_wait_us":157,"spinlock_wait_cycles":7424,"thread_start_us":107,"threads_started":1}
I20260806 18:35:08.987656 31316 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:35:08.981124 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58768 (local address 127.19.194.2:38277)
0806 18:35:08.981975 (+ 851us) server_negotiation.cc:206] Beginning negotiation
0806 18:35:08.981988 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0806 18:35:08.982011 (+ 23us) server_negotiation.cc:407] Connection header received
0806 18:35:08.982085 (+ 74us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:35:08.982090 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:35:08.982274 (+ 184us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:35:08.982765 (+ 491us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 18:35:08.982987 (+ 222us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:35:08.984077 (+ 1090us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:35:08.985520 (+ 1443us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:35:08.986091 (+ 571us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:35:08.986382 (+ 291us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:35:08.986559 (+ 177us) server_negotiation.cc:1035] Waiting for connection context
0806 18:35:08.987369 (+ 810us) server_negotiation.cc:299] Negotiation successful
0806 18:35:08.987477 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":733,"thread_start_us":89,"threads_started":1}
I20260806 18:35:08.987982 31317 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:35:08.981461 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33148 (local address 127.19.194.3:42945)
0806 18:35:08.981828 (+ 367us) server_negotiation.cc:206] Beginning negotiation
0806 18:35:08.981983 (+ 155us) server_negotiation.cc:399] Waiting for connection header
0806 18:35:08.982011 (+ 28us) server_negotiation.cc:407] Connection header received
0806 18:35:08.982085 (+ 74us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:35:08.982090 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:35:08.982274 (+ 184us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:35:08.982426 (+ 152us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 18:35:08.982694 (+ 268us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:35:08.983908 (+ 1214us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:35:08.985587 (+ 1679us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:35:08.986378 (+ 791us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:35:08.986505 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:35:08.986661 (+ 156us) server_negotiation.cc:1035] Waiting for connection context
0806 18:35:08.987724 (+ 1063us) server_negotiation.cc:299] Negotiation successful
0806 18:35:08.987829 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":267,"thread_start_us":155,"threads_started":1}
I20260806 18:35:08.990043 30952 tablet_service.cc:1467] Processing CreateTablet for tablet c29885f907cf48ffa29db0ac6d7bbd5d (DEFAULT_TABLE table=test-table [id=d0ad414e40f1400fb78c5c7f895c781b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 18:35:08.990536 31092 tablet_service.cc:1467] Processing CreateTablet for tablet c29885f907cf48ffa29db0ac6d7bbd5d (DEFAULT_TABLE table=test-table [id=d0ad414e40f1400fb78c5c7f895c781b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 18:35:08.990571 31230 tablet_service.cc:1467] Processing CreateTablet for tablet c29885f907cf48ffa29db0ac6d7bbd5d (DEFAULT_TABLE table=test-table [id=d0ad414e40f1400fb78c5c7f895c781b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 18:35:08.991111 30952 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c29885f907cf48ffa29db0ac6d7bbd5d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 18:35:08.991627 31092 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c29885f907cf48ffa29db0ac6d7bbd5d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 18:35:08.991636 31230 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c29885f907cf48ffa29db0ac6d7bbd5d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 18:35:08.998045 31319 tablet_bootstrap.cc:492] T c29885f907cf48ffa29db0ac6d7bbd5d P ec51f83c60694c8eb32b61ddd2e9b936: Bootstrap starting.
I20260806 18:35:08.998716 31321 tablet_bootstrap.cc:492] T c29885f907cf48ffa29db0ac6d7bbd5d P 4d61ee914f7349a08f7894a0f8122234: Bootstrap starting.
I20260806 18:35:08.999557 31320 tablet_bootstrap.cc:492] T c29885f907cf48ffa29db0ac6d7bbd5d P f0dbb4cd1c4f48dc89152e03214c51de: Bootstrap starting.
I20260806 18:35:09.000452 31319 tablet_bootstrap.cc:654] T c29885f907cf48ffa29db0ac6d7bbd5d P ec51f83c60694c8eb32b61ddd2e9b936: Neither blocks nor log segments found. Creating new log.
I20260806 18:35:09.001672 31320 tablet_bootstrap.cc:654] T c29885f907cf48ffa29db0ac6d7bbd5d P f0dbb4cd1c4f48dc89152e03214c51de: Neither blocks nor log segments found. Creating new log.
I20260806 18:35:09.001749 31319 log.cc:824] T c29885f907cf48ffa29db0ac6d7bbd5d P ec51f83c60694c8eb32b61ddd2e9b936: Log is configured to *not* fsync() on all Append() calls
I20260806 18:35:09.003160 31320 log.cc:824] T c29885f907cf48ffa29db0ac6d7bbd5d P f0dbb4cd1c4f48dc89152e03214c51de: Log is configured to *not* fsync() on all Append() calls
I20260806 18:35:09.003855 31319 tablet_bootstrap.cc:492] T c29885f907cf48ffa29db0ac6d7bbd5d P ec51f83c60694c8eb32b61ddd2e9b936: No bootstrap required, opened a new log
I20260806 18:35:09.004071 31319 ts_tablet_manager.cc:1397] T c29885f907cf48ffa29db0ac6d7bbd5d P ec51f83c60694c8eb32b61ddd2e9b936: Time spent bootstrapping tablet: real 0.006s user 0.005s sys 0.000s
I20260806 18:35:09.004290 31321 tablet_bootstrap.cc:654] T c29885f907cf48ffa29db0ac6d7bbd5d P 4d61ee914f7349a08f7894a0f8122234: Neither blocks nor log segments found. Creating new log.
I20260806 18:35:09.005007 31321 log.cc:824] T c29885f907cf48ffa29db0ac6d7bbd5d P 4d61ee914f7349a08f7894a0f8122234: Log is configured to *not* fsync() on all Append() calls
I20260806 18:35:09.005206 31320 tablet_bootstrap.cc:492] T c29885f907cf48ffa29db0ac6d7bbd5d P f0dbb4cd1c4f48dc89152e03214c51de: No bootstrap required, opened a new log
I20260806 18:35:09.005378 31320 ts_tablet_manager.cc:1397] T c29885f907cf48ffa29db0ac6d7bbd5d P f0dbb4cd1c4f48dc89152e03214c51de: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260806 18:35:09.006641 31321 tablet_bootstrap.cc:492] T c29885f907cf48ffa29db0ac6d7bbd5d P 4d61ee914f7349a08f7894a0f8122234: No bootstrap required, opened a new log
I20260806 18:35:09.006824 31321 ts_tablet_manager.cc:1397] T c29885f907cf48ffa29db0ac6d7bbd5d P 4d61ee914f7349a08f7894a0f8122234: Time spent bootstrapping tablet: real 0.008s user 0.004s sys 0.000s
I20260806 18:35:09.007894 31319 raft_consensus.cc:348] T c29885f907cf48ffa29db0ac6d7bbd5d P ec51f83c60694c8eb32b61ddd2e9b936 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ec51f83c60694c8eb32b61ddd2e9b936" member_type: VOTER last_known_addr { host: "127.19.194.1" port: 37047 } } peers { permanent_uuid: "f0dbb4cd1c4f48dc89152e03214c51de" member_type: VOTER last_known_addr { host: "127.19.194.3" port: 42945 } } peers { permanent_uuid: "4d61ee914f7349a08f7894a0f8122234" member_type: VOTER last_known_addr { host: "127.19.194.2" port: 38277 } }
I20260806 18:35:09.008158 31319 raft_consensus.cc:374] T c29885f907cf48ffa29db0ac6d7bbd5d P ec51f83c60694c8eb32b61ddd2e9b936 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 18:35:09.008244 31319 raft_consensus.cc:729] T c29885f907cf48ffa29db0ac6d7bbd5d P ec51f83c60694c8eb32b61ddd2e9b936 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ec51f83c60694c8eb32b61ddd2e9b936, State: Initialized, Role: FOLLOWER
I20260806 18:35:09.008708 31319 consensus_queue.cc:260] T c29885f907cf48ffa29db0ac6d7bbd5d P ec51f83c60694c8eb32b61ddd2e9b936 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ec51f83c60694c8eb32b61ddd2e9b936" member_type: VOTER last_known_addr { host: "127.19.194.1" port: 37047 } } peers { permanent_uuid: "f0dbb4cd1c4f48dc89152e03214c51de" member_type: VOTER last_known_addr { host: "127.19.194.3" port: 42945 } } peers { permanent_uuid: "4d61ee914f7349a08f7894a0f8122234" member_type: VOTER last_known_addr { host: "127.19.194.2" port: 38277 } }
I20260806 18:35:09.009250 31320 raft_consensus.cc:348] T c29885f907cf48ffa29db0ac6d7bbd5d P f0dbb4cd1c4f48dc89152e03214c51de [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ec51f83c60694c8eb32b61ddd2e9b936" member_type: VOTER last_known_addr { host: "127.19.194.1" port: 37047 } } peers { permanent_uuid: "f0dbb4cd1c4f48dc89152e03214c51de" member_type: VOTER last_known_addr { host: "127.19.194.3" port: 42945 } } peers { permanent_uuid: "4d61ee914f7349a08f7894a0f8122234" member_type: VOTER last_known_addr { host: "127.19.194.2" port: 38277 } }
I20260806 18:35:09.009483 31320 raft_consensus.cc:374] T c29885f907cf48ffa29db0ac6d7bbd5d P f0dbb4cd1c4f48dc89152e03214c51de [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 18:35:09.009531 31320 raft_consensus.cc:729] T c29885f907cf48ffa29db0ac6d7bbd5d P f0dbb4cd1c4f48dc89152e03214c51de [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f0dbb4cd1c4f48dc89152e03214c51de, State: Initialized, Role: FOLLOWER
I20260806 18:35:09.009752 31319 ts_tablet_manager.cc:1428] T c29885f907cf48ffa29db0ac6d7bbd5d P ec51f83c60694c8eb32b61ddd2e9b936: Time spent starting tablet: real 0.006s user 0.002s sys 0.004s
I20260806 18:35:09.009938 31320 consensus_queue.cc:260] T c29885f907cf48ffa29db0ac6d7bbd5d P f0dbb4cd1c4f48dc89152e03214c51de [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ec51f83c60694c8eb32b61ddd2e9b936" member_type: VOTER last_known_addr { host: "127.19.194.1" port: 37047 } } peers { permanent_uuid: "f0dbb4cd1c4f48dc89152e03214c51de" member_type: VOTER last_known_addr { host: "127.19.194.3" port: 42945 } } peers { permanent_uuid: "4d61ee914f7349a08f7894a0f8122234" member_type: VOTER last_known_addr { host: "127.19.194.2" port: 38277 } }
I20260806 18:35:09.010131 31021 heartbeater.cc:503] Master 127.19.194.62:41429 was elected leader, sending a full tablet report...
I20260806 18:35:09.010327 31321 raft_consensus.cc:348] T c29885f907cf48ffa29db0ac6d7bbd5d P 4d61ee914f7349a08f7894a0f8122234 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ec51f83c60694c8eb32b61ddd2e9b936" member_type: VOTER last_known_addr { host: "127.19.194.1" port: 37047 } } peers { permanent_uuid: "f0dbb4cd1c4f48dc89152e03214c51de" member_type: VOTER last_known_addr { host: "127.19.194.3" port: 42945 } } peers { permanent_uuid: "4d61ee914f7349a08f7894a0f8122234" member_type: VOTER last_known_addr { host: "127.19.194.2" port: 38277 } }
I20260806 18:35:09.010531 31301 heartbeater.cc:503] Master 127.19.194.62:41429 was elected leader, sending a full tablet report...
I20260806 18:35:09.010556 31321 raft_consensus.cc:374] T c29885f907cf48ffa29db0ac6d7bbd5d P 4d61ee914f7349a08f7894a0f8122234 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 18:35:09.010612 31321 raft_consensus.cc:729] T c29885f907cf48ffa29db0ac6d7bbd5d P 4d61ee914f7349a08f7894a0f8122234 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4d61ee914f7349a08f7894a0f8122234, State: Initialized, Role: FOLLOWER
W20260806 18:35:09.010803 31022 tablet.cc:2369] T c29885f907cf48ffa29db0ac6d7bbd5d P ec51f83c60694c8eb32b61ddd2e9b936: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 18:35:09.011019 31321 consensus_queue.cc:260] T c29885f907cf48ffa29db0ac6d7bbd5d P 4d61ee914f7349a08f7894a0f8122234 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ec51f83c60694c8eb32b61ddd2e9b936" member_type: VOTER last_known_addr { host: "127.19.194.1" port: 37047 } } peers { permanent_uuid: "f0dbb4cd1c4f48dc89152e03214c51de" member_type: VOTER last_known_addr { host: "127.19.194.3" port: 42945 } } peers { permanent_uuid: "4d61ee914f7349a08f7894a0f8122234" member_type: VOTER last_known_addr { host: "127.19.194.2" port: 38277 } }
I20260806 18:35:09.011755 31320 ts_tablet_manager.cc:1428] T c29885f907cf48ffa29db0ac6d7bbd5d P f0dbb4cd1c4f48dc89152e03214c51de: Time spent starting tablet: real 0.006s user 0.001s sys 0.005s
I20260806 18:35:09.011924 31321 ts_tablet_manager.cc:1428] T c29885f907cf48ffa29db0ac6d7bbd5d P 4d61ee914f7349a08f7894a0f8122234: Time spent starting tablet: real 0.005s user 0.006s sys 0.000s
I20260806 18:35:09.012202 31161 heartbeater.cc:503] Master 127.19.194.62:41429 was elected leader, sending a full tablet report...
I20260806 18:35:09.062928 31325 raft_consensus.cc:482] T c29885f907cf48ffa29db0ac6d7bbd5d P ec51f83c60694c8eb32b61ddd2e9b936 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 18:35:09.063104 31325 raft_consensus.cc:504] T c29885f907cf48ffa29db0ac6d7bbd5d P ec51f83c60694c8eb32b61ddd2e9b936 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ec51f83c60694c8eb32b61ddd2e9b936" member_type: VOTER last_known_addr { host: "127.19.194.1" port: 37047 } } peers { permanent_uuid: "f0dbb4cd1c4f48dc89152e03214c51de" member_type: VOTER last_known_addr { host: "127.19.194.3" port: 42945 } } peers { permanent_uuid: "4d61ee914f7349a08f7894a0f8122234" member_type: VOTER last_known_addr { host: "127.19.194.2" port: 38277 } }
I20260806 18:35:09.064234 31325 leader_election.cc:288] T c29885f907cf48ffa29db0ac6d7bbd5d P ec51f83c60694c8eb32b61ddd2e9b936 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f0dbb4cd1c4f48dc89152e03214c51de (127.19.194.3:42945), 4d61ee914f7349a08f7894a0f8122234 (127.19.194.2:38277)
I20260806 18:35:09.068547 31316 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:35:09.064729 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.194.1:41007 (local address 127.19.194.2:38277)
0806 18:35:09.064873 (+ 144us) server_negotiation.cc:206] Beginning negotiation
0806 18:35:09.064878 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 18:35:09.065016 (+ 138us) server_negotiation.cc:407] Connection header received
0806 18:35:09.065167 (+ 151us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:35:09.065170 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:35:09.065237 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:35:09.065322 (+ 85us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 18:35:09.065653 (+ 331us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:35:09.066322 (+ 669us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:35:09.067466 (+ 1144us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:35:09.067990 (+ 524us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:35:09.068122 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:35:09.068178 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0806 18:35:09.068353 (+ 175us) server_negotiation.cc:299] Negotiation successful
0806 18:35:09.068422 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260806 18:35:09.068563 31329 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:35:09.064479 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.194.2:38277 (local address 127.19.194.1:41007)
0806 18:35:09.064895 (+ 416us) negotiation.cc:107] Waiting for socket to connect
0806 18:35:09.064911 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0806 18:35:09.065054 (+ 143us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:35:09.065332 (+ 278us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:35:09.065335 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:35:09.065350 (+ 15us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 18:35:09.065533 (+ 183us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:35:09.065538 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:35:09.066430 (+ 892us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:35:09.066436 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:35:09.067346 (+ 910us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:35:09.067355 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:35:09.068113 (+ 758us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:35:09.068116 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:35:09.068218 (+ 102us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:35:09.068241 (+ 23us) client_negotiation.cc:769] Sending connection context
0806 18:35:09.068337 (+ 96us) client_negotiation.cc:240] Negotiation successful
0806 18:35:09.068406 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":352,"mutex_wait_us":22,"thread_start_us":132,"threads_started":1}
I20260806 18:35:09.068586 31317 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:35:09.064519 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.194.1:51309 (local address 127.19.194.3:42945)
0806 18:35:09.064689 (+ 170us) server_negotiation.cc:206] Beginning negotiation
0806 18:35:09.064695 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 18:35:09.064887 (+ 192us) server_negotiation.cc:407] Connection header received
0806 18:35:09.065069 (+ 182us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:35:09.065072 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:35:09.065124 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:35:09.065205 (+ 81us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 18:35:09.065612 (+ 407us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:35:09.066285 (+ 673us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:35:09.067486 (+ 1201us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:35:09.068008 (+ 522us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:35:09.068144 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:35:09.068199 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0806 18:35:09.068399 (+ 200us) server_negotiation.cc:299] Negotiation successful
0806 18:35:09.068466 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260806 18:35:09.068605 31328 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:35:09.064405 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.194.3:42945 (local address 127.19.194.1:51309)
0806 18:35:09.064782 (+ 377us) negotiation.cc:107] Waiting for socket to connect
0806 18:35:09.064799 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0806 18:35:09.064933 (+ 134us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:35:09.065217 (+ 284us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:35:09.065275 (+ 58us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:35:09.065292 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 18:35:09.065481 (+ 189us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:35:09.065487 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:35:09.066393 (+ 906us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:35:09.066399 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:35:09.067365 (+ 966us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:35:09.067375 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:35:09.068144 (+ 769us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:35:09.068148 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:35:09.068270 (+ 122us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:35:09.068288 (+ 18us) client_negotiation.cc:769] Sending connection context
0806 18:35:09.068371 (+ 83us) client_negotiation.cc:240] Negotiation successful
0806 18:35:09.068443 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":299,"mutex_wait_us":39,"thread_start_us":100,"threads_started":1}
I20260806 18:35:09.069378 31252 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c29885f907cf48ffa29db0ac6d7bbd5d" candidate_uuid: "ec51f83c60694c8eb32b61ddd2e9b936" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f0dbb4cd1c4f48dc89152e03214c51de" is_pre_election: true
I20260806 18:35:09.069430 31113 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c29885f907cf48ffa29db0ac6d7bbd5d" candidate_uuid: "ec51f83c60694c8eb32b61ddd2e9b936" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4d61ee914f7349a08f7894a0f8122234" is_pre_election: true
I20260806 18:35:09.069679 31252 raft_consensus.cc:2454] T c29885f907cf48ffa29db0ac6d7bbd5d P f0dbb4cd1c4f48dc89152e03214c51de [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ec51f83c60694c8eb32b61ddd2e9b936 in term 0.
I20260806 18:35:09.069712 31113 raft_consensus.cc:2454] T c29885f907cf48ffa29db0ac6d7bbd5d P 4d61ee914f7349a08f7894a0f8122234 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ec51f83c60694c8eb32b61ddd2e9b936 in term 0.
I20260806 18:35:09.070154 30906 leader_election.cc:302] T c29885f907cf48ffa29db0ac6d7bbd5d P ec51f83c60694c8eb32b61ddd2e9b936 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 4d61ee914f7349a08f7894a0f8122234, ec51f83c60694c8eb32b61ddd2e9b936, f0dbb4cd1c4f48dc89152e03214c51de; no voters:
I20260806 18:35:09.070427 31325 raft_consensus.cc:2781] T c29885f907cf48ffa29db0ac6d7bbd5d P ec51f83c60694c8eb32b61ddd2e9b936 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 18:35:09.070508 31325 raft_consensus.cc:482] T c29885f907cf48ffa29db0ac6d7bbd5d P ec51f83c60694c8eb32b61ddd2e9b936 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 18:35:09.070539 31325 raft_consensus.cc:3037] T c29885f907cf48ffa29db0ac6d7bbd5d P ec51f83c60694c8eb32b61ddd2e9b936 [term 0 FOLLOWER]: Advancing to term 1
I20260806 18:35:09.071635 31325 raft_consensus.cc:504] T c29885f907cf48ffa29db0ac6d7bbd5d P ec51f83c60694c8eb32b61ddd2e9b936 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ec51f83c60694c8eb32b61ddd2e9b936" member_type: VOTER last_known_addr { host: "127.19.194.1" port: 37047 } } peers { permanent_uuid: "f0dbb4cd1c4f48dc89152e03214c51de" member_type: VOTER last_known_addr { host: "127.19.194.3" port: 42945 } } peers { permanent_uuid: "4d61ee914f7349a08f7894a0f8122234" member_type: VOTER last_known_addr { host: "127.19.194.2" port: 38277 } }
I20260806 18:35:09.072024 31325 leader_election.cc:288] T c29885f907cf48ffa29db0ac6d7bbd5d P ec51f83c60694c8eb32b61ddd2e9b936 [CANDIDATE]: Term 1 election: Requested vote from peers f0dbb4cd1c4f48dc89152e03214c51de (127.19.194.3:42945), 4d61ee914f7349a08f7894a0f8122234 (127.19.194.2:38277)
I20260806 18:35:09.072357 31252 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c29885f907cf48ffa29db0ac6d7bbd5d" candidate_uuid: "ec51f83c60694c8eb32b61ddd2e9b936" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f0dbb4cd1c4f48dc89152e03214c51de"
I20260806 18:35:09.072377 31113 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c29885f907cf48ffa29db0ac6d7bbd5d" candidate_uuid: "ec51f83c60694c8eb32b61ddd2e9b936" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4d61ee914f7349a08f7894a0f8122234"
I20260806 18:35:09.072482 31252 raft_consensus.cc:3037] T c29885f907cf48ffa29db0ac6d7bbd5d P f0dbb4cd1c4f48dc89152e03214c51de [term 0 FOLLOWER]: Advancing to term 1
I20260806 18:35:09.072504 31113 raft_consensus.cc:3037] T c29885f907cf48ffa29db0ac6d7bbd5d P 4d61ee914f7349a08f7894a0f8122234 [term 0 FOLLOWER]: Advancing to term 1
I20260806 18:35:09.073531 31113 raft_consensus.cc:2454] T c29885f907cf48ffa29db0ac6d7bbd5d P 4d61ee914f7349a08f7894a0f8122234 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ec51f83c60694c8eb32b61ddd2e9b936 in term 1.
I20260806 18:35:09.073611 31252 raft_consensus.cc:2454] T c29885f907cf48ffa29db0ac6d7bbd5d P f0dbb4cd1c4f48dc89152e03214c51de [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ec51f83c60694c8eb32b61ddd2e9b936 in term 1.
I20260806 18:35:09.073812 30905 leader_election.cc:302] T c29885f907cf48ffa29db0ac6d7bbd5d P ec51f83c60694c8eb32b61ddd2e9b936 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4d61ee914f7349a08f7894a0f8122234, ec51f83c60694c8eb32b61ddd2e9b936; no voters:
I20260806 18:35:09.074034 31325 raft_consensus.cc:2781] T c29885f907cf48ffa29db0ac6d7bbd5d P ec51f83c60694c8eb32b61ddd2e9b936 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 18:35:09.074268 31325 raft_consensus.cc:686] T c29885f907cf48ffa29db0ac6d7bbd5d P ec51f83c60694c8eb32b61ddd2e9b936 [term 1 LEADER]: Becoming Leader. State: Replica: ec51f83c60694c8eb32b61ddd2e9b936, State: Running, Role: LEADER
I20260806 18:35:09.074594 31325 consensus_queue.cc:237] T c29885f907cf48ffa29db0ac6d7bbd5d P ec51f83c60694c8eb32b61ddd2e9b936 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ec51f83c60694c8eb32b61ddd2e9b936" member_type: VOTER last_known_addr { host: "127.19.194.1" port: 37047 } } peers { permanent_uuid: "f0dbb4cd1c4f48dc89152e03214c51de" member_type: VOTER last_known_addr { host: "127.19.194.3" port: 42945 } } peers { permanent_uuid: "4d61ee914f7349a08f7894a0f8122234" member_type: VOTER last_known_addr { host: "127.19.194.2" port: 38277 } }
I20260806 18:35:09.077952 30825 catalog_manager.cc:5495] T c29885f907cf48ffa29db0ac6d7bbd5d P ec51f83c60694c8eb32b61ddd2e9b936 reported cstate change: term changed from 0 to 1, leader changed from <none> to ec51f83c60694c8eb32b61ddd2e9b936 (127.19.194.1). New cstate: current_term: 1 leader_uuid: "ec51f83c60694c8eb32b61ddd2e9b936" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ec51f83c60694c8eb32b61ddd2e9b936" member_type: VOTER last_known_addr { host: "127.19.194.1" port: 37047 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f0dbb4cd1c4f48dc89152e03214c51de" member_type: VOTER last_known_addr { host: "127.19.194.3" port: 42945 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4d61ee914f7349a08f7894a0f8122234" member_type: VOTER last_known_addr { host: "127.19.194.2" port: 38277 } health_report { overall_health: UNKNOWN } } }
I20260806 18:35:09.106633 31315 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:35:09.103379 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60616 (local address 127.19.194.1:37047)
0806 18:35:09.103533 (+ 154us) server_negotiation.cc:206] Beginning negotiation
0806 18:35:09.103537 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 18:35:09.103551 (+ 14us) server_negotiation.cc:407] Connection header received
0806 18:35:09.103668 (+ 117us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:35:09.103670 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:35:09.103717 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:35:09.103819 (+ 102us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 18:35:09.104032 (+ 213us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:35:09.104502 (+ 470us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:35:09.105298 (+ 796us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:35:09.105496 (+ 198us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:35:09.105569 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:35:09.105868 (+ 299us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 18:35:09.106268 (+ 400us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 18:35:09.106358 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0806 18:35:09.106422 (+ 64us) server_negotiation.cc:299] Negotiation successful
0806 18:35:09.106469 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260806 18:35:09.112106 31113 raft_consensus.cc:1261] T c29885f907cf48ffa29db0ac6d7bbd5d P 4d61ee914f7349a08f7894a0f8122234 [term 1 FOLLOWER]: Refusing update from remote peer ec51f83c60694c8eb32b61ddd2e9b936: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 18:35:09.112120 31252 raft_consensus.cc:1261] T c29885f907cf48ffa29db0ac6d7bbd5d P f0dbb4cd1c4f48dc89152e03214c51de [term 1 FOLLOWER]: Refusing update from remote peer ec51f83c60694c8eb32b61ddd2e9b936: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 18:35:09.112788 31330 consensus_queue.cc:1035] T c29885f907cf48ffa29db0ac6d7bbd5d P ec51f83c60694c8eb32b61ddd2e9b936 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4d61ee914f7349a08f7894a0f8122234" member_type: VOTER last_known_addr { host: "127.19.194.2" port: 38277 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 18:35:09.112949 31325 consensus_queue.cc:1035] T c29885f907cf48ffa29db0ac6d7bbd5d P ec51f83c60694c8eb32b61ddd2e9b936 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f0dbb4cd1c4f48dc89152e03214c51de" member_type: VOTER last_known_addr { host: "127.19.194.3" port: 42945 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 18:35:09.118913 31333 mvcc.cc:205] Tried to move back new op lower bound from 7315625202116689920 to 7315625201972576256. Current Snapshot: MvccSnapshot[applied={T|T < 7315625202116689920}]
I20260806 18:35:09.120858 31335 mvcc.cc:205] Tried to move back new op lower bound from 7315625202116689920 to 7315625201972576256. Current Snapshot: MvccSnapshot[applied={T|T < 7315625202116689920}]
I20260806 18:35:09.127857 30825 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:51284:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 18:35:09.128029 30825 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:51284:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 18:35:09.130337 30825 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 040776b2955740a28506c9a36d2b6424: Sending DeleteTablet for 3 replicas of tablet c29885f907cf48ffa29db0ac6d7bbd5d
I20260806 18:35:09.131065 30952 tablet_service.cc:1514] Processing DeleteTablet for tablet c29885f907cf48ffa29db0ac6d7bbd5d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 18:35:09 UTC) from {username='kudu', principal='kudu/127.19.194.62@KRBTEST.COM'} at 127.0.0.1:60614
I20260806 18:35:09.131057 31230 tablet_service.cc:1514] Processing DeleteTablet for tablet c29885f907cf48ffa29db0ac6d7bbd5d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 18:35:09 UTC) from {username='kudu', principal='kudu/127.19.194.62@KRBTEST.COM'} at 127.0.0.1:33148
I20260806 18:35:09.131232 31092 tablet_service.cc:1514] Processing DeleteTablet for tablet c29885f907cf48ffa29db0ac6d7bbd5d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 18:35:09 UTC) from {username='kudu', principal='kudu/127.19.194.62@KRBTEST.COM'} at 127.0.0.1:58768
I20260806 18:35:09.131666 31347 tablet_replica.cc:330] T c29885f907cf48ffa29db0ac6d7bbd5d P 4d61ee914f7349a08f7894a0f8122234: stopping tablet replica
I20260806 18:35:09.131670 20232 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJW69tn/build/debug/bin/kudu with pid 30889
I20260806 18:35:09.131986 31347 raft_consensus.cc:2229] T c29885f907cf48ffa29db0ac6d7bbd5d P 4d61ee914f7349a08f7894a0f8122234 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 18:35:09.132290 31347 raft_consensus.cc:2258] T c29885f907cf48ffa29db0ac6d7bbd5d P 4d61ee914f7349a08f7894a0f8122234 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 18:35:09.132429 31346 tablet_replica.cc:330] T c29885f907cf48ffa29db0ac6d7bbd5d P f0dbb4cd1c4f48dc89152e03214c51de: stopping tablet replica
I20260806 18:35:09.132792 31346 raft_consensus.cc:2229] T c29885f907cf48ffa29db0ac6d7bbd5d P f0dbb4cd1c4f48dc89152e03214c51de [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 18:35:09.133126 31346 raft_consensus.cc:2258] T c29885f907cf48ffa29db0ac6d7bbd5d P f0dbb4cd1c4f48dc89152e03214c51de [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 18:35:09.133921 31347 ts_tablet_manager.cc:1905] T c29885f907cf48ffa29db0ac6d7bbd5d P 4d61ee914f7349a08f7894a0f8122234: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 18:35:09.134644 31346 ts_tablet_manager.cc:1905] T c29885f907cf48ffa29db0ac6d7bbd5d P f0dbb4cd1c4f48dc89152e03214c51de: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 18:35:09.136958 31347 ts_tablet_manager.cc:1918] T c29885f907cf48ffa29db0ac6d7bbd5d P 4d61ee914f7349a08f7894a0f8122234: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 18:35:09.137072 31347 log.cc:1196] T c29885f907cf48ffa29db0ac6d7bbd5d P 4d61ee914f7349a08f7894a0f8122234: Deleting WAL directory at /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786041246094603-20232-0/minicluster-data/ts-1/wal/wals/c29885f907cf48ffa29db0ac6d7bbd5d
I20260806 18:35:09.137444 31347 ts_tablet_manager.cc:1939] T c29885f907cf48ffa29db0ac6d7bbd5d P 4d61ee914f7349a08f7894a0f8122234: Deleting consensus metadata
I20260806 18:35:09.137480 31346 ts_tablet_manager.cc:1918] T c29885f907cf48ffa29db0ac6d7bbd5d P f0dbb4cd1c4f48dc89152e03214c51de: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 18:35:09.137579 31346 log.cc:1196] T c29885f907cf48ffa29db0ac6d7bbd5d P f0dbb4cd1c4f48dc89152e03214c51de: Deleting WAL directory at /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786041246094603-20232-0/minicluster-data/ts-2/wal/wals/c29885f907cf48ffa29db0ac6d7bbd5d
I20260806 18:35:09.137903 31346 ts_tablet_manager.cc:1939] T c29885f907cf48ffa29db0ac6d7bbd5d P f0dbb4cd1c4f48dc89152e03214c51de: Deleting consensus metadata
I20260806 18:35:09.138672 30808 catalog_manager.cc:4841] TS 4d61ee914f7349a08f7894a0f8122234 (127.19.194.2:38277): tablet c29885f907cf48ffa29db0ac6d7bbd5d (table test-table [id=d0ad414e40f1400fb78c5c7f895c781b]) successfully deleted
I20260806 18:35:09.138993 30809 catalog_manager.cc:4841] TS f0dbb4cd1c4f48dc89152e03214c51de (127.19.194.3:42945): tablet c29885f907cf48ffa29db0ac6d7bbd5d (table test-table [id=d0ad414e40f1400fb78c5c7f895c781b]) successfully deleted
W20260806 18:35:09.141495 30811 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.19.194.1:37047 (error 108)
I20260806 18:35:09.142239 20232 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJW69tn/build/debug/bin/kudu with pid 31027
I20260806 18:35:09.142423 31314 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:35:09.142120 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.194.1:37047 (local address 127.0.0.1:60622)
0806 18:35:09.142253 (+ 133us) negotiation.cc:107] Waiting for socket to connect
0806 18:35:09.142331 (+ 78us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.194.1:37047: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":49}
W20260806 18:35:09.143997 30811 catalog_manager.cc:4584] TS ec51f83c60694c8eb32b61ddd2e9b936 (127.19.194.1:37047): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet c29885f907cf48ffa29db0ac6d7bbd5d: Network error: Client connection negotiation failed: client connection to 127.19.194.1:37047: connect: Connection refused (error 111)
I20260806 18:35:09.148777 20232 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJW69tn/build/debug/bin/kudu with pid 31166
I20260806 18:35:09.155851 20232 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJW69tn/build/debug/bin/kudu with pid 30792
2026-08-06T18:35:09Z chronyd exiting
[ OK ] SecurityITest.TestRequireAuthenticationSecureCluster (2295 ms)
[ RUN ] SecurityITest.TestEncryptionWithKMSIntegration
Loading random data
Initializing database '/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786041246094603-20232-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 18:35:09 dist-test-slave-1xrh krb5kdc[31351](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 18:35:09 dist-test-slave-1xrh krb5kdc[31351](info): set up 2 sockets
Aug 06 18:35:09 dist-test-slave-1xrh krb5kdc[31351](info): commencing operation
krb5kdc: starting...
W20260806 18:35:10.217734 20232 mini_kdc.cc:121] Time spent starting KDC: real 1.039s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786041246094603-20232-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786041246094603-20232-0/krb5kdc/test-user.keytab.
Aug 06 18:35:10 dist-test-slave-1xrh krb5kdc[31351](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041310, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T18:35:10Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T18:35:10Z Disabled control of system clock
WARNING: no policy specified for rangeradmin/127.19.194.20@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangeradmin/127.19.194.20@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangeradmin/127.19.194.20@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786041246094603-20232-0/krb5kdc/rangeradmin_127.19.194.20@KRBTEST.COM.keytab.
Entry for principal rangeradmin/127.19.194.20@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786041246094603-20232-0/krb5kdc/rangeradmin_127.19.194.20@KRBTEST.COM.keytab.
WARNING: no policy specified for rangerlookup/127.19.194.20@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerlookup/127.19.194.20@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerlookup/127.19.194.20@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786041246094603-20232-0/krb5kdc/rangerlookup_127.19.194.20@KRBTEST.COM.keytab.
Entry for principal rangerlookup/127.19.194.20@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786041246094603-20232-0/krb5kdc/rangerlookup_127.19.194.20@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.19.194.20@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "HTTP/127.19.194.20@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.19.194.20@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786041246094603-20232-0/krb5kdc/HTTP_127.19.194.20@KRBTEST.COM.keytab.
Entry for principal HTTP/127.19.194.20@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786041246094603-20232-0/krb5kdc/HTTP_127.19.194.20@KRBTEST.COM.keytab.
I20260806 18:35:10.389447 20232 mini_postgres.cc:62] Running initdb...
The files belonging to this database system will be owned by user "slave".
This user must also own the server process.
The database cluster will be initialized with locale "C".
The default database encoding has accordingly been set to "SQL_ASCII".
The default text search configuration will be set to "english".
Data page checksums are disabled.
creating directory /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786041246094603-20232-0/minicluster-data/postgres ... ok
creating subdirectories ... ok
selecting dynamic shared memory implementation ... posix
selecting default max_connections ... 100
selecting default shared_buffers ... 128MB
selecting default time zone ... Etc/UTC
creating configuration files ... ok
running bootstrap script ... ok
performing post-bootstrap initialization ... ok
syncing data to disk ... ok
initdb: warning: enabling "trust" authentication for local connections
You can change this by editing pg_hba.conf or using the option -A, or
--auth-local and --auth-host, the next time you run initdb.
Success. You can now start the database server using:
/tmp/dist-test-taskJW69tn/build/debug/bin/postgres/pg_ctl -D /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786041246094603-20232-0/minicluster-data/postgres -l logfile start
2026-08-06 18:35:12.872 UTC [31382] LOG: starting PostgreSQL 12.2 on x86_64-pc-linux-gnu, compiled by cc (GCC) 8.3.1 20190311 (Red Hat 8.3.1-3), 64-bit
2026-08-06 18:35:12.872 UTC [31382] LOG: listening on IPv4 address "127.19.194.20", port 56911
2026-08-06 18:35:12.875 UTC [31382] LOG: listening on Unix socket "/tmp/.s.PGSQL.56911"
2026-08-06 18:35:12.896 UTC [31385] LOG: database system was shut down at 2026-08-06 18:35:11 UTC
2026-08-06 18:35:12.900 UTC [31382] LOG: database system is ready to accept connections
I20260806 18:35:13.868801 20232 mini_postgres.cc:96] Postgres bound to 56911
2026-08-06 18:35:13.877 UTC [31393] FATAL: database "slave" does not exist
127.19.194.20:56911 - accepting connections
I20260806 18:35:13.878206 20232 mini_ranger.cc:161] Starting Ranger...
I20260806 18:35:13.894312 20232 mini_ranger.cc:84] Created miniranger Postgres user
I20260806 18:35:14.367494 20232 mini_ranger.cc:87] Created ranger Postgres database
I20260806 18:35:14.367612 20232 mini_ranger.cc:178] Starting Ranger out of /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786041246094603-20232-0/minicluster-data/ranger-admin
2026-08-06 18:35:14,672 [I] DB FLAVOR :POSTGRES
2026-08-06 18:35:14,672 [I] --------- Verifying Ranger DB connection ---------
2026-08-06 18:35:14,672 [I] Checking connection..
2026-08-06 18:35:14,672 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskJW69tn/build/debug/bin/postgresql.jar:/tmp/dist-test-taskJW69tn/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.19.194.20:56911/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select 1;"
2026-08-06 18:35:14,911 [I] Checking connection passed.
2026-08-06 18:35:14,912 [I] --------- Verifying version history table ---------
2026-08-06 18:35:14,912 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskJW69tn/build/debug/bin/postgresql.jar:/tmp/dist-test-taskJW69tn/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.19.194.20:56911/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-06 18:35:15,158 [I] Table x_db_version_h does not exist in database ranger
2026-08-06 18:35:15,159 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskJW69tn/build/debug/bin/postgresql.jar:/tmp/dist-test-taskJW69tn/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.19.194.20:56911/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-06 18:35:15,411 [I] Table x_db_version_h does not exist in database ranger
2026-08-06 18:35:15,411 [I] Importing x_db_version_h table schema to database ranger from file: create_dbversion_catalog.sql
2026-08-06 18:35:15,411 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskJW69tn/build/debug/bin/postgresql.jar:/tmp/dist-test-taskJW69tn/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.19.194.20:56911/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskJW69tn/thirdparty/src/ranger-2.1.0-admin/db/postgres/create_dbversion_catalog.sql
2026-08-06 18:35:15.650 UTC [31460] WARNING: there is no transaction in progress
2026-08-06 18:35:15,657 [I] create_dbversion_catalog.sql file imported successfully
2026-08-06 18:35:15,658 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskJW69tn/build/debug/bin/postgresql.jar:/tmp/dist-test-taskJW69tn/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.19.194.20:56911/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-06 18:35:15,909 [I] Table x_db_version_h already exists in database 'ranger'
2026-08-06 18:35:15,909 [I] --------- Importing Ranger Core DB Schema ---------
2026-08-06 18:35:15,910 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskJW69tn/build/debug/bin/postgresql.jar:/tmp/dist-test-taskJW69tn/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.19.194.20:56911/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'Y';"
2026-08-06 18:35:16,167 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskJW69tn/build/debug/bin/postgresql.jar:/tmp/dist-test-taskJW69tn/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.19.194.20:56911/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'N';"
2026-08-06 18:35:16,400 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskJW69tn/build/debug/bin/postgresql.jar:/tmp/dist-test-taskJW69tn/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.19.194.20:56911/ranger -u miniranger -p '********' -noheader -trim -c \; -query "insert into x_db_version_h (version, inst_at, inst_by, updated_at, updated_by,active) values ('CORE_DB_SCHEMA', current_timestamp, 'Ranger 2.1.0', current_timestamp, 'dist-test-slave-1xrh.c.gcp-upstream.internal','N') ;"
2026-08-06 18:35:16,631 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskJW69tn/build/debug/bin/postgresql.jar:/tmp/dist-test-taskJW69tn/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.19.194.20:56911/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-08-06 18:35:16,889 [I] Table x_portal_user does not exist in database ranger
2026-08-06 18:35:16,890 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskJW69tn/build/debug/bin/postgresql.jar:/tmp/dist-test-taskJW69tn/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.19.194.20:56911/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-08-06 18:35:17,141 [I] Table x_policy_ref_group does not exist in database ranger
2026-08-06 18:35:17,142 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskJW69tn/build/debug/bin/postgresql.jar:/tmp/dist-test-taskJW69tn/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.19.194.20:56911/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-08-06 18:35:17,397 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskJW69tn/build/debug/bin/postgresql.jar:/tmp/dist-test-taskJW69tn/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.19.194.20:56911/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-08-06 18:35:17,652 [I] Importing DB schema to database ranger from file: ranger_core_db_postgres.sql
2026-08-06 18:35:17,652 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskJW69tn/build/debug/bin/postgresql.jar:/tmp/dist-test-taskJW69tn/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.19.194.20:56911/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskJW69tn/thirdparty/src/ranger-2.1.0-admin/db/postgres/optimized/current/ranger_core_db_postgres.sql
2026-08-06 18:35:18.569 UTC [31574] WARNING: there is no transaction in progress
2026-08-06 18:35:18.584 UTC [31574] WARNING: there is no transaction in progress
2026-08-06 18:35:18.599 UTC [31574] WARNING: there is no transaction in progress
2026-08-06 18:35:18.613 UTC [31574] WARNING: there is no transaction in progress
2026-08-06 18:35:18.628 UTC [31574] WARNING: there is no transaction in progress
2026-08-06 18:35:18.643 UTC [31574] WARNING: there is no transaction in progress
2026-08-06 18:35:18.691 UTC [31574] WARNING: there is no transaction in progress
2026-08-06 18:35:18.697 UTC [31574] WARNING: there is no transaction in progress
2026-08-06 18:35:18.711 UTC [31574] WARNING: there is no transaction in progress
2026-08-06 18:35:18.720 UTC [31574] WARNING: there is no transaction in progress
2026-08-06 18:35:18.731 UTC [31574] WARNING: there is no transaction in progress
2026-08-06 18:35:18.744 UTC [31574] WARNING: there is no transaction in progress
2026-08-06 18:35:18.753 UTC [31574] WARNING: there is no transaction in progress
2026-08-06 18:35:18.759 UTC [31574] WARNING: there is no transaction in progress
2026-08-06 18:35:18.766 UTC [31574] WARNING: there is no transaction in progress
2026-08-06 18:35:18.773 UTC [31574] WARNING: there is no transaction in progress
2026-08-06 18:35:19,374 [I] ranger_core_db_postgres.sql file imported successfully
2026-08-06 18:35:19,374 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskJW69tn/build/debug/bin/postgresql.jar:/tmp/dist-test-taskJW69tn/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.19.194.20:56911/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set inst_by='Ranger 2.1.0' where active='Y' and updated_by='localhost';"
2026-08-06 18:35:19,614 [I] Patches status entries updated from base ranger version to current installed ranger version:Ranger 2.1.0
2026-08-06 18:35:19,614 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskJW69tn/build/debug/bin/postgresql.jar:/tmp/dist-test-taskJW69tn/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.19.194.20:56911/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-08-06 18:35:19,855 [I] Table x_portal_user already exists in database 'ranger'
2026-08-06 18:35:19,855 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskJW69tn/build/debug/bin/postgresql.jar:/tmp/dist-test-taskJW69tn/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.19.194.20:56911/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-08-06 18:35:20,103 [I] Table x_policy_ref_group already exists in database 'ranger'
2026-08-06 18:35:20,103 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskJW69tn/build/debug/bin/postgresql.jar:/tmp/dist-test-taskJW69tn/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.19.194.20:56911/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-08-06 18:35:20,363 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskJW69tn/build/debug/bin/postgresql.jar:/tmp/dist-test-taskJW69tn/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.19.194.20:56911/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-08-06 18:35:20,604 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskJW69tn/build/debug/bin/postgresql.jar:/tmp/dist-test-taskJW69tn/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.19.194.20:56911/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set active='Y' where version='CORE_DB_SCHEMA' and active='N' and updated_by='dist-test-slave-1xrh.c.gcp-upstream.internal';"
2026-08-06 18:35:20,853 [I] CORE_DB_SCHEMA import status has been updated
2026-08-06 18:35:20,853 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskJW69tn/build/debug/bin/postgresql.jar:/tmp/dist-test-taskJW69tn/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.19.194.20:56911/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and inst_by = 'Ranger 2.1.0' and active = 'Y';"
2026-08-06 18:35:21,122 [I] DB_PATCHES have already been applied
I20260806 18:35:21.128425 20232 mini_ranger.cc:191] Using Ranger class path: /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786041246094603-20232-0/minicluster-data/ranger-admin:/tmp/dist-test-taskJW69tn/thirdparty/src/ranger-2.1.0-admin/ews/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*:/tmp/dist-test-taskJW69tn/thirdparty/src/hadoop-3.2.0/*:/tmp/dist-test-taskJW69tn/build/debug/bin/postgresql.jar:/tmp/dist-test-taskJW69tn/thirdparty/src/ranger-2.1.0-admin/ews/webapp
I20260806 18:35:21.128492 20232 mini_ranger.cc:193] Using host: 127.19.194.20
I20260806 18:35:21.132517 20232 mini_ranger.cc:227] Ranger admin URL: http://127.19.194.20:46047
Aug 06, 2026 6:35:21 PM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Aug 06, 2026 6:35:21 PM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Aug 06, 2026 6:35:21 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Deriving webapp folder from catalina.base property. folder=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786041246094603-20232-0/minicluster-data/ranger-admin/ews/webapp
Aug 06, 2026 6:35:21 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786041246094603-20232-0/minicluster-data/ranger-admin/ews/webapp, webAppName = /
Aug 06, 2026 6:35:21 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/] = path [/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786041246094603-20232-0/minicluster-data/ranger-admin/ews/webapp] .....
Aug 06, 2026 6:35:21 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/] = path [/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786041246094603-20232-0/minicluster-data/ranger-admin/ews/webapp].
Aug 06, 2026 6:35:21 PM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : miniranger
I20260806 18:35:22.287849 20232 mini_ranger.cc:160] Time spent starting Ranger: real 8.410s user 0.000s sys 0.012s
Aug 06, 2026 6:35:42 PM com.sun.jersey.api.core.PackagesResourceConfig init
INFO: Scanning for root resource and provider classes in the packages:
org.apache.ranger.rest
org.apache.ranger.common
xa.rest
Aug 06, 2026 6:35:42 PM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Root resource classes found:
class org.apache.ranger.rest.PublicAPIsv2
class org.apache.ranger.rest.XUserREST
class org.apache.ranger.rest.TagREST
class org.apache.ranger.rest.PublicAPIs
class org.apache.ranger.rest.SecurityZoneREST
class org.apache.ranger.rest.XAuditREST
class org.apache.ranger.rest.UserREST
class org.apache.ranger.rest.XKeyREST
class org.apache.ranger.rest.RoleREST
class org.apache.ranger.rest.AssetREST
class org.apache.ranger.rest.ServiceREST
class org.apache.ranger.rest.MetricsREST
Aug 06, 2026 6:35:42 PM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Provider classes found:
class org.apache.ranger.common.RangerJAXBContextResolver
Aug 06, 2026 6:35:42 PM com.sun.jersey.spi.spring.container.servlet.SpringServlet getContext
INFO: Using default applicationContext
Aug 06, 2026 6:35:42 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, assetREST, of type org.apache.ranger.rest.AssetREST as a root resource class
Aug 06, 2026 6:35:42 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, metricsREST, of type org.apache.ranger.rest.MetricsREST as a root resource class
Aug 06, 2026 6:35:42 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIs, of type org.apache.ranger.rest.PublicAPIs as a root resource class
Aug 06, 2026 6:35:42 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIsv2, of type org.apache.ranger.rest.PublicAPIsv2 as a root resource class
Aug 06, 2026 6:35:42 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, roleREST, of type org.apache.ranger.rest.RoleREST as a root resource class
Aug 06, 2026 6:35:42 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, securityZoneREST, of type org.apache.ranger.rest.SecurityZoneREST as a root resource class
Aug 06, 2026 6:35:42 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, serviceREST, of type org.apache.ranger.rest.ServiceREST as a root resource class
Aug 06, 2026 6:35:42 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, tagREST, of type org.apache.ranger.rest.TagREST as a root resource class
Aug 06, 2026 6:35:42 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, userREST, of type org.apache.ranger.rest.UserREST as a root resource class
Aug 06, 2026 6:35:42 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XAuditREST, of type org.apache.ranger.rest.XAuditREST as a root resource class
Aug 06, 2026 6:35:42 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XKeyREST, of type org.apache.ranger.rest.XKeyREST as a root resource class
Aug 06, 2026 6:35:42 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XUserREST, of type org.apache.ranger.rest.XUserREST as a root resource class
Aug 06, 2026 6:35:42 PM com.sun.jersey.server.impl.application.WebApplicationImpl _initiate
INFO: Initiating Jersey application, version 'Jersey: 1.19.3 10/24/2016 03:58 PM'
Aug 06, 2026 6:35:43 PM com.sun.jersey.spi.inject.Errors processErrorMessages
WARNING: The following warnings have been detected with resource and/or provider classes:
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInExcel(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInJson(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse,java.lang.Boolean), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInCsv(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse) throws java.io.IOException, MUST return a non-void type.
I20260806 18:35:44.292539 20232 mini_ranger.cc:261] Created Kudu service
WARNING: no policy specified for rangerkms/127.19.194.20@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerkms/127.19.194.20@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerkms/127.19.194.20@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786041246094603-20232-0/krb5kdc/rangerkms_127.19.194.20@KRBTEST.COM.keytab.
Entry for principal rangerkms/127.19.194.20@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786041246094603-20232-0/krb5kdc/rangerkms_127.19.194.20@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.19.194.20@KRBTEST.COM; defaulting to no policy
add_principal: Principal or policy already exists while creating "HTTP/127.19.194.20@KRBTEST.COM".
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.19.194.20@KRBTEST.COM with kvno 3, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786041246094603-20232-0/krb5kdc/HTTP_127.19.194.20@KRBTEST.COM.keytab.
Entry for principal HTTP/127.19.194.20@KRBTEST.COM with kvno 3, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786041246094603-20232-0/krb5kdc/HTTP_127.19.194.20@KRBTEST.COM.keytab.
WARNING: no policy specified for keyadmin@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "keyadmin@KRBTEST.COM" created.
Aug 06 18:35:44 dist-test-slave-1xrh krb5kdc[31351](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041344, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for keyadmin@KRBTEST.COM:
I20260806 18:35:44.390865 20232 mini_ranger_kms.cc:208] Starting Ranger KMS...
I20260806 18:35:44.401216 20232 mini_ranger_kms.cc:78] Created minirangerkms Postgres user
I20260806 18:35:45.326014 20232 mini_ranger_kms.cc:81] Created rangerkms Postgres database
I20260806 18:35:45.326124 20232 mini_ranger_kms.cc:226] Starting Ranger KMS out of /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786041246094603-20232-0/minicluster-data/ranger-kms
I20260806 18:35:45.326243 20232 mini_ranger_kms.cc:227] Using postgres at 127.19.194.20:56911
2026-08-06 18:35:45,551 [I] DB FLAVOR :POSTGRES
2026-08-06 18:35:45,551 [I] --------- Verifying Ranger DB connection ---------
2026-08-06 18:35:45,551 [I] Checking connection
2026-08-06 18:35:45,551 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskJW69tn/build/debug/bin/postgresql.jar:/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786041246094603-20232-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.19.194.20:56911/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "SELECT 1;"
2026-08-06 18:35:45,795 [I] connection success
2026-08-06 18:35:45,796 [I] --------- Verifying Ranger DB tables ---------
2026-08-06 18:35:45,796 [I] Verifying table ranger_masterkey in database rangerkms
2026-08-06 18:35:45,796 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskJW69tn/build/debug/bin/postgresql.jar:/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786041246094603-20232-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.19.194.20:56911/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='rangerkms' and table_name = 'ranger_masterkey') as temp;"
2026-08-06 18:35:46,024 [I] Table ranger_masterkey does not exist in database rangerkms
2026-08-06 18:35:46,025 [I] --------- Importing Ranger Core DB Schema ---------
2026-08-06 18:35:46,025 [I] Importing db schema to database rangerkms from file: kms_core_db_postgres.sql
2026-08-06 18:35:46,025 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskJW69tn/build/debug/bin/postgresql.jar:/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786041246094603-20232-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.19.194.20:56911/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786041246094603-20232-0/minicluster-data/ranger-kms/db/postgres/kms_core_db_postgres.sql
2026-08-06 18:35:46,284 [I] kms_core_db_postgres.sql DB schema imported successfully
I20260806 18:35:46.760967 20232 mini_ranger_kms.cc:310] Created kms service
I20260806 18:35:46.942389 20232 mini_ranger_kms.cc:326] Created kudu user
I20260806 18:35:46.986027 20232 mini_ranger_kms.cc:343] Created rangerkms user
Aug 06 18:35:46 dist-test-slave-1xrh krb5kdc[31351](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041344, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for HTTP/127.19.194.20@KRBTEST.COM
I20260806 18:35:47.375646 20232 mini_ranger_kms.cc:384] Added ranger policy
I20260806 18:35:47.375809 20232 mini_ranger_kms.cc:238] Using RangerKMS classpath: /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786041246094603-20232-0/minicluster-data/ranger-kms:/tmp/dist-test-taskJW69tn/build/debug/bin/postgresql.jar:/tmp/dist-test-taskJW69tn/thirdparty/src/ranger-2.1.0-kms/ews/webapp/WEB-INF/classes/lib/*:/tmp/dist-test-taskJW69tn/thirdparty/src/ranger-2.1.0-kms/ews/webapp/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*
I20260806 18:35:47.375833 20232 mini_ranger_kms.cc:240] Using host: 127.19.194.20
I20260806 18:35:47.380313 20232 mini_ranger_kms.cc:276] Ranger KMS PID: 31762
I20260806 18:35:47.380450 20232 mini_ranger_kms.cc:277] Ranger KMS URL: http://127.19.194.20:33783
Aug 06, 2026 6:35:48 PM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Aug 06, 2026 6:35:48 PM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Aug 06, 2026 6:35:48 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786041246094603-20232-0/minicluster-data/ranger-kms/ews/webapp, webAppName = /kms
Aug 06, 2026 6:35:48 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/kms] = path [/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786041246094603-20232-0/minicluster-data/ranger-kms/ews/webapp] .....
Aug 06, 2026 6:35:49 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/kms] = path [/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786041246094603-20232-0/minicluster-data/ranger-kms/ews/webapp].
Aug 06, 2026 6:35:49 PM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : minirangerkms
I20260806 18:35:49.759953 20232 mini_ranger_kms.cc:207] Time spent starting Ranger KMS: real 5.369s user 0.001s sys 0.209s
I20260806 18:35:49.760106 20232 mini_ranger_kms.cc:397] {"name":"kuduclusterkey","cipher":"AES/CTR/NoPadding","length":128,"description":"kuduclusterkey"}
I20260806 18:35:49.760239 20232 mini_ranger_kms.cc:401] 127.19.194.20:33783/kms/v1/keys
Aug 06 18:35:49 dist-test-slave-1xrh krb5kdc[31351](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786041349, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.19.194.20@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Aug 06 18:35:50 dist-test-slave-1xrh krb5kdc[31351](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041349, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.19.194.20@KRBTEST.COM for HTTP/127.19.194.20@KRBTEST.COM
Aug 06 18:35:50 dist-test-slave-1xrh krb5kdc[31351](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041349, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.19.194.20@KRBTEST.COM for HTTP/127.19.194.20@KRBTEST.COM
Aug 06 18:35:52 dist-test-slave-1xrh krb5kdc[31351](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041352, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
WARNING: no policy specified for kudu/127.19.194.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.19.194.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.19.194.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786041246094603-20232-0/krb5kdc/kudu_127.19.194.62.keytab.
Entry for principal kudu/127.19.194.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786041246094603-20232-0/krb5kdc/kudu_127.19.194.62.keytab.
I20260806 18:35:53.018122 20232 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJW69tn/build/debug/bin/kudu
/tmp/dist-test-taskJW69tn/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786041246094603-20232-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786041246094603-20232-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786041246094603-20232-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786041246094603-20232-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.19.194.62:41165
--webserver_interface=127.19.194.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786041246094603-20232-0/krb5kdc/kudu_127.19.194.62.keytab
--principal=kudu/127.19.194.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.19.194.20:35145
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.19.194.62:41165
--ranger_config_path=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786041246094603-20232-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.19.194.20:33783/kms
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786041246094603-20232-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786041246094603-20232-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786041246094603-20232-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 18:35:53.144021 31815 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 18:35:53.144290 31815 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 18:35:53.144342 31815 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 18:35:53.148881 31815 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 18:35:53.148939 31815 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 18:35:53.148960 31815 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 18:35:53.148978 31815 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 18:35:53.148995 31815 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 18:35:53.154950 31815 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.194.20:35145
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786041246094603-20232-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786041246094603-20232-0/minicluster-data/master-0/wal
--ranger_kms_url=127.19.194.20:33783/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.19.194.62:41165
--ranger_config_path=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786041246094603-20232-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786041246094603-20232-0/krb5kdc/kudu_127.19.194.62.keytab
--principal=kudu/127.19.194.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.19.194.62:41165
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786041246094603-20232-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.19.194.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31815
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786041246094603-20232-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 17:59:13 UTC on ccycloud.quasar-wvsnzx.root.comops.site
build id 4571028
I20260806 18:35:53.156080 31815 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 18:35:53.157172 31815 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 18:35:53.166277 31821 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:35:53.166260 31820 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:35:53.166260 31823 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 18:35:53.166622 31815 server_base.cc:1034] running on GCE node
I20260806 18:35:53.167340 31815 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 18:35:53.168479 31815 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 18:35:53.169689 31815 hybrid_clock.cc:648] HybridClock initialized: now 1786041353169667 us; error 39 us; skew 500 ppm
Aug 06 18:35:53 dist-test-slave-1xrh krb5kdc[31351](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041353, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.194.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 18:35:53.172783 31815 init.cc:377] Logged in from keytab as kudu/127.19.194.62@KRBTEST.COM (short username kudu)
I20260806 18:35:53.173735 31815 webserver.cc:460] Webserver started at http://127.19.194.62:37139/ using document root <none> and password file <none>
I20260806 18:35:53.174300 31815 fs_manager.cc:359] Metadata directory not provided
I20260806 18:35:53.174361 31815 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 18:35:53.174587 31815 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 06 18:35:53 dist-test-slave-1xrh krb5kdc[31351](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041353, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.194.62@KRBTEST.COM for HTTP/127.19.194.20@KRBTEST.COM
I20260806 18:35:53.227267 31815 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786041246094603-20232-0/minicluster-data/master-0/data/instance:
uuid: "e05daed5b6f74deeabdfff19b89c06ee"
format_stamp: "Formatted at 2026-08-06 18:35:53 on dist-test-slave-1xrh"
server_key: "c052ca59a4371ab0f67740d02d8f8877"
server_key_iv: "a31ac6e4d2e26bbe74102b0a4023f9ac"
server_key_version: "kuduclusterkey@0"
I20260806 18:35:53.228004 31815 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786041246094603-20232-0/minicluster-data/master-0/wal/instance:
uuid: "e05daed5b6f74deeabdfff19b89c06ee"
format_stamp: "Formatted at 2026-08-06 18:35:53 on dist-test-slave-1xrh"
server_key: "c052ca59a4371ab0f67740d02d8f8877"
server_key_iv: "a31ac6e4d2e26bbe74102b0a4023f9ac"
server_key_version: "kuduclusterkey@0"
I20260806 18:35:53.233083 31815 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260806 18:35:53.247174 31832 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 18:35:53.248567 31815 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260806 18:35:53.248709 31815 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786041246094603-20232-0/minicluster-data/master-0/data,/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786041246094603-20232-0/minicluster-data/master-0/wal
uuid: "e05daed5b6f74deeabdfff19b89c06ee"
format_stamp: "Formatted at 2026-08-06 18:35:53 on dist-test-slave-1xrh"
server_key: "c052ca59a4371ab0f67740d02d8f8877"
server_key_iv: "a31ac6e4d2e26bbe74102b0a4023f9ac"
server_key_version: "kuduclusterkey@0"
I20260806 18:35:53.248857 31815 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786041246094603-20232-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786041246094603-20232-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786041246094603-20232-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 18:35:53.264444 31815 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 18:35:53.267422 31815 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 18:35:53.267642 31815 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 18:35:53.275624 31815 rpc_server.cc:307] RPC server started. Bound to: 127.19.194.62:41165
I20260806 18:35:53.275650 31887 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.194.62:41165 every 8 connection(s)
I20260806 18:35:53.276584 31815 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786041246094603-20232-0/minicluster-data/master-0/data/info.pb
I20260806 18:35:53.277068 20232 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJW69tn/build/debug/bin/kudu as pid 31815
I20260806 18:35:53.277215 20232 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786041246094603-20232-0/minicluster-data/master-0/wal/instance
I20260806 18:35:53.277509 20232 external_mini_cluster.cc:1428] Setting key ea78e0738e1d309adc5d6afa07a5a25d
I20260806 18:35:53.280020 31888 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
Aug 06 18:35:53 dist-test-slave-1xrh krb5kdc[31351](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041352, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.19.194.62@KRBTEST.COM
I20260806 18:35:53.286875 31888 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e05daed5b6f74deeabdfff19b89c06ee: Bootstrap starting.
I20260806 18:35:53.290031 31888 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P e05daed5b6f74deeabdfff19b89c06ee: Neither blocks nor log segments found. Creating new log.
I20260806 18:35:53.292311 31888 log.cc:824] T 00000000000000000000000000000000 P e05daed5b6f74deeabdfff19b89c06ee: Log is configured to *not* fsync() on all Append() calls
I20260806 18:35:53.293279 31891 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:35:53.279111 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42592 (local address 127.19.194.62:41165)
0806 18:35:53.279627 (+ 516us) server_negotiation.cc:206] Beginning negotiation
0806 18:35:53.279642 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0806 18:35:53.279675 (+ 33us) server_negotiation.cc:407] Connection header received
0806 18:35:53.280392 (+ 717us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:35:53.280408 (+ 16us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:35:53.280706 (+ 298us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:35:53.281056 (+ 350us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:35:53.281881 (+ 825us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:35:53.282848 (+ 967us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:35:53.283743 (+ 895us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:35:53.284230 (+ 487us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:35:53.284399 (+ 169us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:35:53.286854 (+ 2455us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:35:53.286860 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:35:53.286871 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 18:35:53.286895 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:35:53.288550 (+ 1655us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:35:53.289226 (+ 676us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:35:53.289233 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:35:53.289237 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:35:53.289389 (+ 152us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:35:53.289839 (+ 450us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:35:53.289845 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:35:53.289849 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:35:53.290520 (+ 671us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:35:53.290877 (+ 357us) server_negotiation.cc:1035] Waiting for connection context
0806 18:35:53.291476 (+ 599us) server_negotiation.cc:299] Negotiation successful
0806 18:35:53.291971 (+ 495us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":329,"thread_start_us":145,"threads_started":1}
I20260806 18:35:53.295647 31888 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e05daed5b6f74deeabdfff19b89c06ee: No bootstrap required, opened a new log
I20260806 18:35:53.301236 31888 raft_consensus.cc:348] T 00000000000000000000000000000000 P e05daed5b6f74deeabdfff19b89c06ee [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e05daed5b6f74deeabdfff19b89c06ee" member_type: VOTER last_known_addr { host: "127.19.194.62" port: 41165 } }
I20260806 18:35:53.301597 31888 raft_consensus.cc:374] T 00000000000000000000000000000000 P e05daed5b6f74deeabdfff19b89c06ee [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 18:35:53.301683 31888 raft_consensus.cc:729] T 00000000000000000000000000000000 P e05daed5b6f74deeabdfff19b89c06ee [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e05daed5b6f74deeabdfff19b89c06ee, State: Initialized, Role: FOLLOWER
I20260806 18:35:53.302368 31888 consensus_queue.cc:260] T 00000000000000000000000000000000 P e05daed5b6f74deeabdfff19b89c06ee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e05daed5b6f74deeabdfff19b89c06ee" member_type: VOTER last_known_addr { host: "127.19.194.62" port: 41165 } }
I20260806 18:35:53.302556 31888 raft_consensus.cc:388] T 00000000000000000000000000000000 P e05daed5b6f74deeabdfff19b89c06ee [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 18:35:53.302688 31888 raft_consensus.cc:482] T 00000000000000000000000000000000 P e05daed5b6f74deeabdfff19b89c06ee [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 18:35:53.302870 31888 raft_consensus.cc:3037] T 00000000000000000000000000000000 P e05daed5b6f74deeabdfff19b89c06ee [term 0 FOLLOWER]: Advancing to term 1
I20260806 18:35:53.304780 31888 raft_consensus.cc:504] T 00000000000000000000000000000000 P e05daed5b6f74deeabdfff19b89c06ee [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e05daed5b6f74deeabdfff19b89c06ee" member_type: VOTER last_known_addr { host: "127.19.194.62" port: 41165 } }
I20260806 18:35:53.305259 31888 leader_election.cc:302] T 00000000000000000000000000000000 P e05daed5b6f74deeabdfff19b89c06ee [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: e05daed5b6f74deeabdfff19b89c06ee; no voters:
I20260806 18:35:53.305722 31888 leader_election.cc:288] T 00000000000000000000000000000000 P e05daed5b6f74deeabdfff19b89c06ee [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 18:35:53.305872 31893 raft_consensus.cc:2781] T 00000000000000000000000000000000 P e05daed5b6f74deeabdfff19b89c06ee [term 1 FOLLOWER]: Leader election won for term 1
I20260806 18:35:53.306242 31893 raft_consensus.cc:686] T 00000000000000000000000000000000 P e05daed5b6f74deeabdfff19b89c06ee [term 1 LEADER]: Becoming Leader. State: Replica: e05daed5b6f74deeabdfff19b89c06ee, State: Running, Role: LEADER
I20260806 18:35:53.306769 31893 consensus_queue.cc:237] T 00000000000000000000000000000000 P e05daed5b6f74deeabdfff19b89c06ee [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e05daed5b6f74deeabdfff19b89c06ee" member_type: VOTER last_known_addr { host: "127.19.194.62" port: 41165 } }
I20260806 18:35:53.307850 31888 sys_catalog.cc:564] T 00000000000000000000000000000000 P e05daed5b6f74deeabdfff19b89c06ee [sys.catalog]: configured and running, proceeding with master startup.
I20260806 18:35:53.309336 31893 sys_catalog.cc:455] T 00000000000000000000000000000000 P e05daed5b6f74deeabdfff19b89c06ee [sys.catalog]: SysCatalogTable state changed. Reason: New leader e05daed5b6f74deeabdfff19b89c06ee. Latest consensus state: current_term: 1 leader_uuid: "e05daed5b6f74deeabdfff19b89c06ee" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e05daed5b6f74deeabdfff19b89c06ee" member_type: VOTER last_known_addr { host: "127.19.194.62" port: 41165 } } }
I20260806 18:35:53.309584 31893 sys_catalog.cc:458] T 00000000000000000000000000000000 P e05daed5b6f74deeabdfff19b89c06ee [sys.catalog]: This master's current role is: LEADER
I20260806 18:35:53.309769 31894 sys_catalog.cc:455] T 00000000000000000000000000000000 P e05daed5b6f74deeabdfff19b89c06ee [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "e05daed5b6f74deeabdfff19b89c06ee" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e05daed5b6f74deeabdfff19b89c06ee" member_type: VOTER last_known_addr { host: "127.19.194.62" port: 41165 } } }
I20260806 18:35:53.309928 31894 sys_catalog.cc:458] T 00000000000000000000000000000000 P e05daed5b6f74deeabdfff19b89c06ee [sys.catalog]: This master's current role is: LEADER
I20260806 18:35:53.310287 31900 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 18:35:53.317540 31900 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 18:35:53.318219 31888 ranger_client.cc:318] Using new properties file: /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786041246094603-20232-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
I20260806 18:35:53.336408 31900 catalog_manager.cc:1347] Generated new cluster ID: 3b50b7d632ea4fbbb7578ca6c5ffd610
I20260806 18:35:53.336573 31900 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 18:35:53.347467 31900 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 18:35:53.348959 31900 catalog_manager.cc:1504] Loading token signing keys...
I20260806 18:35:53.362566 31900 catalog_manager.cc:5868] T 00000000000000000000000000000000 P e05daed5b6f74deeabdfff19b89c06ee: Generated new TSK 0
I20260806 18:35:53.363687 31900 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.19.194.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.19.194.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.19.194.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786041246094603-20232-0/krb5kdc/kudu_127.19.194.1.keytab.
Entry for principal kudu/127.19.194.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786041246094603-20232-0/krb5kdc/kudu_127.19.194.1.keytab.
I20260806 18:35:54.129982 20232 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJW69tn/build/debug/bin/kudu
/tmp/dist-test-taskJW69tn/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786041246094603-20232-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786041246094603-20232-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786041246094603-20232-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786041246094603-20232-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.19.194.20:33783/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.19.194.1:0
--local_ip_for_outbound_sockets=127.19.194.1
--webserver_interface=127.19.194.1
--webserver_port=0
--tserver_master_addrs=127.19.194.62:41165
--keytab_file=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786041246094603-20232-0/krb5kdc/kudu_127.19.194.1.keytab
--principal=kudu/127.19.194.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.19.194.20:35145
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786041246094603-20232-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786041246094603-20232-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786041246094603-20232-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 18:35:54.258458 31934 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 18:35:54.258720 31934 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 18:35:54.258800 31934 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 18:35:54.262694 31934 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 18:35:54.262779 31934 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 18:35:54.262867 31934 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.19.194.1
I20260806 18:35:54.268000 31934 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.194.20:35145
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786041246094603-20232-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786041246094603-20232-0/minicluster-data/ts-0/wal
--ranger_kms_url=127.19.194.20:33783/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786041246094603-20232-0/krb5kdc/kudu_127.19.194.1.keytab
--principal=kudu/127.19.194.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.19.194.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786041246094603-20232-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.19.194.1
--webserver_port=0
--tserver_master_addrs=127.19.194.62:41165
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31934
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.19.194.1
--log_dir=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786041246094603-20232-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 17:59:13 UTC on ccycloud.quasar-wvsnzx.root.comops.site
build id 4571028
I20260806 18:35:54.268987 31934 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 18:35:54.270005 31934 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 18:35:54.280689 31941 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 18:35:54.280855 31934 server_base.cc:1034] running on GCE node
W20260806 18:35:54.281497 31943 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:35:54.281522 31940 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 18:35:54.282084 31934 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 18:35:54.282800 31934 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 18:35:54.284003 31934 hybrid_clock.cc:648] HybridClock initialized: now 1786041354283985 us; error 54 us; skew 500 ppm
Aug 06 18:35:54 dist-test-slave-1xrh krb5kdc[31351](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041354, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.194.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 18:35:54.287484 31934 init.cc:377] Logged in from keytab as kudu/127.19.194.1@KRBTEST.COM (short username kudu)
I20260806 18:35:54.288507 31934 webserver.cc:460] Webserver started at http://127.19.194.1:33257/ using document root <none> and password file <none>
I20260806 18:35:54.289206 31934 fs_manager.cc:359] Metadata directory not provided
I20260806 18:35:54.289319 31934 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 18:35:54.289592 31934 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 06 18:35:54 dist-test-slave-1xrh krb5kdc[31351](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041354, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.194.1@KRBTEST.COM for HTTP/127.19.194.20@KRBTEST.COM
I20260806 18:35:54.305821 31934 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786041246094603-20232-0/minicluster-data/ts-0/data/instance:
uuid: "01b2b8b4f9d7433490cddb51513a986f"
format_stamp: "Formatted at 2026-08-06 18:35:54 on dist-test-slave-1xrh"
server_key: "06a7aa4e48cad658e81064fe3479a7d1"
server_key_iv: "e2a27792e379d0e72d3cc6aae7c39175"
server_key_version: "kuduclusterkey@0"
I20260806 18:35:54.306565 31934 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786041246094603-20232-0/minicluster-data/ts-0/wal/instance:
uuid: "01b2b8b4f9d7433490cddb51513a986f"
format_stamp: "Formatted at 2026-08-06 18:35:54 on dist-test-slave-1xrh"
server_key: "06a7aa4e48cad658e81064fe3479a7d1"
server_key_iv: "e2a27792e379d0e72d3cc6aae7c39175"
server_key_version: "kuduclusterkey@0"
I20260806 18:35:54.312055 31934 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260806 18:35:54.331511 31952 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 18:35:54.333400 31934 fs_manager.cc:725] Time spent opening block manager: real 0.005s user 0.002s sys 0.001s
I20260806 18:35:54.333525 31934 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786041246094603-20232-0/minicluster-data/ts-0/data,/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786041246094603-20232-0/minicluster-data/ts-0/wal
uuid: "01b2b8b4f9d7433490cddb51513a986f"
format_stamp: "Formatted at 2026-08-06 18:35:54 on dist-test-slave-1xrh"
server_key: "06a7aa4e48cad658e81064fe3479a7d1"
server_key_iv: "e2a27792e379d0e72d3cc6aae7c39175"
server_key_version: "kuduclusterkey@0"
I20260806 18:35:54.333645 31934 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786041246094603-20232-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786041246094603-20232-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786041246094603-20232-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 18:35:54.355274 31934 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 18:35:54.358436 31934 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 18:35:54.358644 31934 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 18:35:54.359280 31934 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 18:35:54.360121 31934 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 18:35:54.360194 31934 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 18:35:54.360312 31934 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 18:35:54.360375 31934 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 18:35:54.371243 31934 rpc_server.cc:307] RPC server started. Bound to: 127.19.194.1:42923
I20260806 18:35:54.372453 31934 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786041246094603-20232-0/minicluster-data/ts-0/data/info.pb
I20260806 18:35:54.378137 20232 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJW69tn/build/debug/bin/kudu as pid 31934
I20260806 18:35:54.378283 20232 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786041246094603-20232-0/minicluster-data/ts-0/wal/instance
I20260806 18:35:54.378645 20232 external_mini_cluster.cc:1428] Setting key 2c8d806462e0fc72c23a4ed41e538dfb
I20260806 18:35:54.383816 32068 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.194.1:42923 every 8 connection(s)
WARNING: no policy specified for kudu/127.19.194.2@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.19.194.2@KRBTEST.COM" created.
Aug 06 18:35:54 dist-test-slave-1xrh krb5kdc[31351](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041354, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.194.1@KRBTEST.COM for kudu/127.19.194.62@KRBTEST.COM
I20260806 18:35:54.406229 32073 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:35:54.386245 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.194.62:41165 (local address 127.19.194.1:50253)
0806 18:35:54.388052 (+ 1807us) negotiation.cc:107] Waiting for socket to connect
0806 18:35:54.388106 (+ 54us) client_negotiation.cc:174] Beginning negotiation
0806 18:35:54.389096 (+ 990us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:35:54.391767 (+ 2671us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:35:54.391774 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:35:54.392103 (+ 329us) client_negotiation.cc:189] Negotiated authn=SASL
0806 18:35:54.392580 (+ 477us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:35:54.392594 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:35:54.393578 (+ 984us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:35:54.393582 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:35:54.395887 (+ 2305us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:35:54.395903 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:35:54.396911 (+ 1008us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:35:54.396917 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:35:54.397153 (+ 236us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:35:54.398682 (+ 1529us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 18:35:54.398698 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0806 18:35:54.400430 (+ 1732us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 18:35:54.402907 (+ 2477us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:35:54.402914 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:35:54.402930 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:35:54.403318 (+ 388us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:35:54.403652 (+ 334us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:35:54.403655 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:35:54.403658 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:35:54.403845 (+ 187us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:35:54.404346 (+ 501us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 18:35:54.404352 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 18:35:54.404650 (+ 298us) client_negotiation.cc:769] Sending connection context
0806 18:35:54.404852 (+ 202us) client_negotiation.cc:240] Negotiation successful
0806 18:35:54.405177 (+ 325us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1577,"thread_start_us":149,"threads_started":1}
I20260806 18:35:54.407008 32072 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:35:54.385679 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.194.1:50253 (local address 127.19.194.62:41165)
0806 18:35:54.391311 (+ 5632us) server_negotiation.cc:206] Beginning negotiation
0806 18:35:54.391319 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 18:35:54.391336 (+ 17us) server_negotiation.cc:407] Connection header received
0806 18:35:54.391387 (+ 51us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:35:54.391393 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:35:54.391454 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:35:54.391573 (+ 119us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:35:54.392739 (+ 1166us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:35:54.393448 (+ 709us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:35:54.396060 (+ 2612us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:35:54.396461 (+ 401us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:35:54.396574 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:35:54.400638 (+ 4064us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:35:54.400643 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:35:54.400646 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 18:35:54.400674 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:35:54.402764 (+ 2090us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:35:54.403455 (+ 691us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:35:54.403459 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:35:54.403461 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:35:54.403554 (+ 93us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:35:54.403955 (+ 401us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:35:54.403959 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:35:54.403961 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:35:54.404186 (+ 225us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:35:54.404280 (+ 94us) server_negotiation.cc:1035] Waiting for connection context
0806 18:35:54.406694 (+ 2414us) server_negotiation.cc:299] Negotiation successful
0806 18:35:54.406833 (+ 139us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":5532,"thread_start_us":92,"threads_started":1}
I20260806 18:35:54.408342 32069 heartbeater.cc:347] Connected to a master server at 127.19.194.62:41165
I20260806 18:35:54.408600 32069 heartbeater.cc:464] Registering TS with master...
I20260806 18:35:54.409157 32069 heartbeater.cc:511] Master 127.19.194.62:41165 requested a full tablet report, sending...
I20260806 18:35:54.411092 31849 ts_manager.cc:194] Registered new tserver with Master: 01b2b8b4f9d7433490cddb51513a986f (127.19.194.1:42923)
I20260806 18:35:54.412662 31849 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.19.194.1@KRBTEST.COM'} at 127.19.194.1:50253
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.19.194.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786041246094603-20232-0/krb5kdc/kudu_127.19.194.2.keytab.
Entry for principal kudu/127.19.194.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786041246094603-20232-0/krb5kdc/kudu_127.19.194.2.keytab.
I20260806 18:35:54.423220 20232 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJW69tn/build/debug/bin/kudu
/tmp/dist-test-taskJW69tn/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786041246094603-20232-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786041246094603-20232-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786041246094603-20232-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786041246094603-20232-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.19.194.20:33783/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.19.194.2:0
--local_ip_for_outbound_sockets=127.19.194.2
--webserver_interface=127.19.194.2
--webserver_port=0
--tserver_master_addrs=127.19.194.62:41165
--keytab_file=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786041246094603-20232-0/krb5kdc/kudu_127.19.194.2.keytab
--principal=kudu/127.19.194.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.19.194.20:35145
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786041246094603-20232-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786041246094603-20232-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786041246094603-20232-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 06 18:35:54 dist-test-slave-1xrh krb5kdc[31351](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786041354, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.194.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
W20260806 18:35:54.584091 32077 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 18:35:54.584421 32077 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 18:35:54.584486 32077 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 18:35:54.590361 32077 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 18:35:54.590466 32077 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 18:35:54.590560 32077 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.19.194.2
I20260806 18:35:54.598026 32077 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.194.20:35145
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786041246094603-20232-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786041246094603-20232-0/minicluster-data/ts-1/wal
--ranger_kms_url=127.19.194.20:33783/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786041246094603-20232-0/krb5kdc/kudu_127.19.194.2.keytab
--principal=kudu/127.19.194.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.19.194.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786041246094603-20232-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.19.194.2
--webserver_port=0
--tserver_master_addrs=127.19.194.62:41165
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.32077
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.19.194.2
--log_dir=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786041246094603-20232-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 17:59:13 UTC on ccycloud.quasar-wvsnzx.root.comops.site
build id 4571028
I20260806 18:35:54.599440 32077 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 18:35:54.600760 32077 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 18:35:54.615379 32084 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:35:54.615767 32086 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:35:54.616532 32083 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 18:35:54.617184 32077 server_base.cc:1034] running on GCE node
I20260806 18:35:54.617800 32077 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 18:35:54.618561 32077 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 18:35:54.619711 32077 hybrid_clock.cc:648] HybridClock initialized: now 1786041354619650 us; error 77 us; skew 500 ppm
Aug 06 18:35:54 dist-test-slave-1xrh krb5kdc[31351](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041354, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.194.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 18:35:54.623278 32077 init.cc:377] Logged in from keytab as kudu/127.19.194.2@KRBTEST.COM (short username kudu)
I20260806 18:35:54.624310 32077 webserver.cc:460] Webserver started at http://127.19.194.2:35203/ using document root <none> and password file <none>
I20260806 18:35:54.624974 32077 fs_manager.cc:359] Metadata directory not provided
I20260806 18:35:54.625058 32077 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 18:35:54.625258 32077 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 06 18:35:54 dist-test-slave-1xrh krb5kdc[31351](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041354, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.194.2@KRBTEST.COM for HTTP/127.19.194.20@KRBTEST.COM
I20260806 18:35:54.645891 32077 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786041246094603-20232-0/minicluster-data/ts-1/data/instance:
uuid: "838e3fc117634b71b4073ada1f1524d1"
format_stamp: "Formatted at 2026-08-06 18:35:54 on dist-test-slave-1xrh"
server_key: "7f110b490a178021f07a6f8a6559f228"
server_key_iv: "8c8ce677a64c3ad727aacd65d60e7820"
server_key_version: "kuduclusterkey@0"
I20260806 18:35:54.646740 32077 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786041246094603-20232-0/minicluster-data/ts-1/wal/instance:
uuid: "838e3fc117634b71b4073ada1f1524d1"
format_stamp: "Formatted at 2026-08-06 18:35:54 on dist-test-slave-1xrh"
server_key: "7f110b490a178021f07a6f8a6559f228"
server_key_iv: "8c8ce677a64c3ad727aacd65d60e7820"
server_key_version: "kuduclusterkey@0"
I20260806 18:35:54.650894 32077 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.000s sys 0.003s
I20260806 18:35:54.668182 32093 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 18:35:54.669713 32077 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20260806 18:35:54.669828 32077 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786041246094603-20232-0/minicluster-data/ts-1/data,/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786041246094603-20232-0/minicluster-data/ts-1/wal
uuid: "838e3fc117634b71b4073ada1f1524d1"
format_stamp: "Formatted at 2026-08-06 18:35:54 on dist-test-slave-1xrh"
server_key: "7f110b490a178021f07a6f8a6559f228"
server_key_iv: "8c8ce677a64c3ad727aacd65d60e7820"
server_key_version: "kuduclusterkey@0"
I20260806 18:35:54.669941 32077 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786041246094603-20232-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786041246094603-20232-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786041246094603-20232-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 18:35:54.704064 32077 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 18:35:54.707760 32077 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 18:35:54.707942 32077 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 18:35:54.708602 32077 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 18:35:54.709666 32077 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 18:35:54.709728 32077 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 18:35:54.709781 32077 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 18:35:54.709846 32077 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.001s sys 0.000s
I20260806 18:35:54.720548 32077 rpc_server.cc:307] RPC server started. Bound to: 127.19.194.2:40377
I20260806 18:35:54.720633 32215 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.194.2:40377 every 8 connection(s)
I20260806 18:35:54.721673 32077 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786041246094603-20232-0/minicluster-data/ts-1/data/info.pb
I20260806 18:35:54.728824 20232 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJW69tn/build/debug/bin/kudu as pid 32077
I20260806 18:35:54.728987 20232 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786041246094603-20232-0/minicluster-data/ts-1/wal/instance
I20260806 18:35:54.729259 20232 external_mini_cluster.cc:1428] Setting key 553b2163203daa0bda5045a04f73d802
Aug 06 18:35:54 dist-test-slave-1xrh krb5kdc[31351](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041354, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.194.2@KRBTEST.COM for kudu/127.19.194.62@KRBTEST.COM
I20260806 18:35:54.735925 32072 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:35:54.723624 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.194.2:33859 (local address 127.19.194.62:41165)
0806 18:35:54.723764 (+ 140us) server_negotiation.cc:206] Beginning negotiation
0806 18:35:54.723768 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 18:35:54.724829 (+ 1061us) server_negotiation.cc:407] Connection header received
0806 18:35:54.725756 (+ 927us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:35:54.725759 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:35:54.725816 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:35:54.725925 (+ 109us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:35:54.726986 (+ 1061us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:35:54.727592 (+ 606us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:35:54.728845 (+ 1253us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:35:54.729204 (+ 359us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:35:54.729335 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:35:54.732193 (+ 2858us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:35:54.732202 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:35:54.732204 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 18:35:54.732227 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:35:54.733764 (+ 1537us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:35:54.734273 (+ 509us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:35:54.734277 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:35:54.734279 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:35:54.734364 (+ 85us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:35:54.734700 (+ 336us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:35:54.734705 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:35:54.734706 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:35:54.734887 (+ 181us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:35:54.734998 (+ 111us) server_negotiation.cc:1035] Waiting for connection context
0806 18:35:54.735635 (+ 637us) server_negotiation.cc:299] Negotiation successful
0806 18:35:54.735769 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260806 18:35:54.737166 32218 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:35:54.724149 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.194.62:41165 (local address 127.19.194.2:33859)
0806 18:35:54.724669 (+ 520us) negotiation.cc:107] Waiting for socket to connect
0806 18:35:54.724717 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0806 18:35:54.725543 (+ 826us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:35:54.726099 (+ 556us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:35:54.726106 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:35:54.726375 (+ 269us) client_negotiation.cc:189] Negotiated authn=SASL
0806 18:35:54.726835 (+ 460us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:35:54.726847 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:35:54.727727 (+ 880us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:35:54.727731 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:35:54.728697 (+ 966us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:35:54.728709 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:35:54.729386 (+ 677us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:35:54.729391 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:35:54.729582 (+ 191us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:35:54.730578 (+ 996us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 18:35:54.730615 (+ 37us) client_negotiation.cc:656] calling sasl_client_start()
0806 18:35:54.732026 (+ 1411us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 18:35:54.733888 (+ 1862us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:35:54.733895 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:35:54.733915 (+ 20us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:35:54.734164 (+ 249us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:35:54.734459 (+ 295us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:35:54.734463 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:35:54.734466 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:35:54.734601 (+ 135us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:35:54.734997 (+ 396us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 18:35:54.735004 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 18:35:54.735353 (+ 349us) client_negotiation.cc:769] Sending connection context
0806 18:35:54.735565 (+ 212us) client_negotiation.cc:240] Negotiation successful
0806 18:35:54.735914 (+ 349us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":333,"thread_start_us":169,"threads_started":1}
I20260806 18:35:54.738669 32216 heartbeater.cc:347] Connected to a master server at 127.19.194.62:41165
I20260806 18:35:54.738938 32216 heartbeater.cc:464] Registering TS with master...
I20260806 18:35:54.739552 32216 heartbeater.cc:511] Master 127.19.194.62:41165 requested a full tablet report, sending...
I20260806 18:35:54.740753 31849 ts_manager.cc:194] Registered new tserver with Master: 838e3fc117634b71b4073ada1f1524d1 (127.19.194.2:40377)
WARNING: no policy specified for kudu/127.19.194.3@KRBTEST.COM; defaulting to no policy
I20260806 18:35:54.741593 31849 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.19.194.2@KRBTEST.COM'} at 127.19.194.2:33859
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.19.194.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.19.194.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786041246094603-20232-0/krb5kdc/kudu_127.19.194.3.keytab.
Entry for principal kudu/127.19.194.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786041246094603-20232-0/krb5kdc/kudu_127.19.194.3.keytab.
I20260806 18:35:54.770830 20232 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJW69tn/build/debug/bin/kudu
/tmp/dist-test-taskJW69tn/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786041246094603-20232-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786041246094603-20232-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786041246094603-20232-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786041246094603-20232-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.19.194.20:33783/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.19.194.3:0
--local_ip_for_outbound_sockets=127.19.194.3
--webserver_interface=127.19.194.3
--webserver_port=0
--tserver_master_addrs=127.19.194.62:41165
--keytab_file=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786041246094603-20232-0/krb5kdc/kudu_127.19.194.3.keytab
--principal=kudu/127.19.194.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.19.194.20:35145
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786041246094603-20232-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786041246094603-20232-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786041246094603-20232-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 18:35:54.902586 32221 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 18:35:54.902992 32221 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 18:35:54.903147 32221 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 18:35:54.909255 32221 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 18:35:54.909384 32221 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 18:35:54.909531 32221 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.19.194.3
Aug 06 18:35:54 dist-test-slave-1xrh krb5kdc[31351](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041354, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.194.62@KRBTEST.COM for HTTP/127.19.194.20@KRBTEST.COM
I20260806 18:35:54.917146 32221 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.194.20:35145
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786041246094603-20232-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786041246094603-20232-0/minicluster-data/ts-2/wal
--ranger_kms_url=127.19.194.20:33783/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786041246094603-20232-0/krb5kdc/kudu_127.19.194.3.keytab
--principal=kudu/127.19.194.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.19.194.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786041246094603-20232-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.19.194.3
--webserver_port=0
--tserver_master_addrs=127.19.194.62:41165
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.32221
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.19.194.3
--log_dir=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786041246094603-20232-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 17:59:13 UTC on ccycloud.quasar-wvsnzx.root.comops.site
build id 4571028
I20260806 18:35:54.918596 32221 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 18:35:54.919971 32221 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 18:35:54.937088 32226 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:35:54.937358 32227 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:35:54.938376 32230 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 18:35:54.940099 32221 server_base.cc:1034] running on GCE node
I20260806 18:35:54.940625 32221 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 18:35:54.941478 32221 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 18:35:54.950814 32221 hybrid_clock.cc:648] HybridClock initialized: now 1786041354950738 us; error 80 us; skew 500 ppm
Aug 06 18:35:54 dist-test-slave-1xrh krb5kdc[31351](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041354, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.194.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 18:35:54.954725 32221 init.cc:377] Logged in from keytab as kudu/127.19.194.3@KRBTEST.COM (short username kudu)
I20260806 18:35:54.955812 32221 webserver.cc:460] Webserver started at http://127.19.194.3:32959/ using document root <none> and password file <none>
I20260806 18:35:54.956544 32221 fs_manager.cc:359] Metadata directory not provided
I20260806 18:35:54.956640 32221 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 18:35:54.956890 32221 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 06 18:35:54 dist-test-slave-1xrh krb5kdc[31351](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041354, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.194.3@KRBTEST.COM for HTTP/127.19.194.20@KRBTEST.COM
I20260806 18:35:54.973990 32221 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786041246094603-20232-0/minicluster-data/ts-2/data/instance:
uuid: "95d8c1822545422193c5994cc716f6e7"
format_stamp: "Formatted at 2026-08-06 18:35:54 on dist-test-slave-1xrh"
server_key: "511dafc8aa3c2cf8b7c13f1d5af0d7e7"
server_key_iv: "2af3a404510d04b1b84a0bac5713627e"
server_key_version: "kuduclusterkey@0"
I20260806 18:35:54.983261 32221 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786041246094603-20232-0/minicluster-data/ts-2/wal/instance:
uuid: "95d8c1822545422193c5994cc716f6e7"
format_stamp: "Formatted at 2026-08-06 18:35:54 on dist-test-slave-1xrh"
server_key: "511dafc8aa3c2cf8b7c13f1d5af0d7e7"
server_key_iv: "2af3a404510d04b1b84a0bac5713627e"
server_key_version: "kuduclusterkey@0"
I20260806 18:35:54.992640 32221 fs_manager.cc:691] Time spent creating directory manager: real 0.009s user 0.004s sys 0.000s
I20260806 18:35:55.032418 32237 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 18:35:55.034124 32221 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.001s sys 0.001s
I20260806 18:35:55.034310 32221 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786041246094603-20232-0/minicluster-data/ts-2/data,/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786041246094603-20232-0/minicluster-data/ts-2/wal
uuid: "95d8c1822545422193c5994cc716f6e7"
format_stamp: "Formatted at 2026-08-06 18:35:54 on dist-test-slave-1xrh"
server_key: "511dafc8aa3c2cf8b7c13f1d5af0d7e7"
server_key_iv: "2af3a404510d04b1b84a0bac5713627e"
server_key_version: "kuduclusterkey@0"
I20260806 18:35:55.034502 32221 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786041246094603-20232-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786041246094603-20232-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786041246094603-20232-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 18:35:55.067301 32221 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 18:35:55.070456 32221 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 18:35:55.070678 32221 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 18:35:55.071324 32221 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 18:35:55.072161 32221 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 18:35:55.072234 32221 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 18:35:55.072304 32221 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 18:35:55.072355 32221 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 18:35:55.082370 32221 rpc_server.cc:307] RPC server started. Bound to: 127.19.194.3:44499
I20260806 18:35:55.083552 32221 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786041246094603-20232-0/minicluster-data/ts-2/data/info.pb
I20260806 18:35:55.083549 32362 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.194.3:44499 every 8 connection(s)
I20260806 18:35:55.085700 20232 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJW69tn/build/debug/bin/kudu as pid 32221
I20260806 18:35:55.085812 20232 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786041246094603-20232-0/minicluster-data/ts-2/wal/instance
I20260806 18:35:55.086095 20232 external_mini_cluster.cc:1428] Setting key 7b3785e2801606d29deb153770dafdcd
Aug 06 18:35:55 dist-test-slave-1xrh krb5kdc[31351](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041354, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.194.3@KRBTEST.COM for kudu/127.19.194.62@KRBTEST.COM
I20260806 18:35:55.098062 32072 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:35:55.086840 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.194.3:42025 (local address 127.19.194.62:41165)
0806 18:35:55.086959 (+ 119us) server_negotiation.cc:206] Beginning negotiation
0806 18:35:55.086963 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 18:35:55.087674 (+ 711us) server_negotiation.cc:407] Connection header received
0806 18:35:55.088690 (+ 1016us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:35:55.088693 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:35:55.088744 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:35:55.088845 (+ 101us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:35:55.089854 (+ 1009us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:35:55.090398 (+ 544us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:35:55.091627 (+ 1229us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:35:55.091854 (+ 227us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:35:55.091965 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:35:55.094453 (+ 2488us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:35:55.094455 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:35:55.094457 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 18:35:55.094479 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:35:55.096055 (+ 1576us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:35:55.096578 (+ 523us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:35:55.096582 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:35:55.096584 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:35:55.096665 (+ 81us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:35:55.097015 (+ 350us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:35:55.097019 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:35:55.097021 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:35:55.097197 (+ 176us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:35:55.097302 (+ 105us) server_negotiation.cc:1035] Waiting for connection context
0806 18:35:55.097759 (+ 457us) server_negotiation.cc:299] Negotiation successful
0806 18:35:55.097883 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260806 18:35:55.099086 32365 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:35:55.086179 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.194.62:41165 (local address 127.19.194.3:42025)
0806 18:35:55.087525 (+ 1346us) negotiation.cc:107] Waiting for socket to connect
0806 18:35:55.087578 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0806 18:35:55.088501 (+ 923us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:35:55.089013 (+ 512us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:35:55.089019 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:35:55.089266 (+ 247us) client_negotiation.cc:189] Negotiated authn=SASL
0806 18:35:55.089718 (+ 452us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:35:55.089730 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:35:55.090565 (+ 835us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:35:55.090567 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:35:55.091499 (+ 932us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:35:55.091508 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:35:55.092018 (+ 510us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:35:55.092022 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:35:55.092182 (+ 160us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:35:55.092820 (+ 638us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 18:35:55.092856 (+ 36us) client_negotiation.cc:656] calling sasl_client_start()
0806 18:35:55.094258 (+ 1402us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 18:35:55.096191 (+ 1933us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:35:55.096197 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:35:55.096211 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:35:55.096472 (+ 261us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:35:55.096758 (+ 286us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:35:55.096761 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:35:55.096763 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:35:55.096907 (+ 144us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:35:55.097301 (+ 394us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 18:35:55.097308 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 18:35:55.097539 (+ 231us) client_negotiation.cc:769] Sending connection context
0806 18:35:55.097717 (+ 178us) client_negotiation.cc:240] Negotiation successful
0806 18:35:55.098002 (+ 285us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":972,"thread_start_us":140,"threads_started":1}
I20260806 18:35:55.100374 32363 heartbeater.cc:347] Connected to a master server at 127.19.194.62:41165
I20260806 18:35:55.100608 32363 heartbeater.cc:464] Registering TS with master...
I20260806 18:35:55.101081 32363 heartbeater.cc:511] Master 127.19.194.62:41165 requested a full tablet report, sending...
I20260806 18:35:55.102085 31848 ts_manager.cc:194] Registered new tserver with Master: 95d8c1822545422193c5994cc716f6e7 (127.19.194.3:44499)
I20260806 18:35:55.102658 31848 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.19.194.3@KRBTEST.COM'} at 127.19.194.3:42025
I20260806 18:35:55.111984 20232 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260806 18:35:55.121861 32072 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:35:55.114102 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42602 (local address 127.19.194.62:41165)
0806 18:35:55.114228 (+ 126us) server_negotiation.cc:206] Beginning negotiation
0806 18:35:55.114233 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 18:35:55.114553 (+ 320us) server_negotiation.cc:407] Connection header received
0806 18:35:55.114686 (+ 133us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:35:55.114688 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:35:55.114737 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:35:55.114818 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:35:55.115640 (+ 822us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:35:55.116151 (+ 511us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:35:55.116977 (+ 826us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:35:55.117203 (+ 226us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:35:55.117312 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:35:55.118697 (+ 1385us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:35:55.118752 (+ 55us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:35:55.118754 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 18:35:55.118774 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:35:55.120225 (+ 1451us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:35:55.120653 (+ 428us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:35:55.120657 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:35:55.120659 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:35:55.120739 (+ 80us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:35:55.121013 (+ 274us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:35:55.121017 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:35:55.121019 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:35:55.121189 (+ 170us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:35:55.121328 (+ 139us) server_negotiation.cc:1035] Waiting for connection context
0806 18:35:55.121563 (+ 235us) server_negotiation.cc:299] Negotiation successful
0806 18:35:55.121673 (+ 110us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260806 18:35:55.124801 31848 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:42602:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260806 18:35:55.127911 31848 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 18:35:55.145937 32380 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:35:55.138555 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.194.2:40377 (local address 127.0.0.1:59036)
0806 18:35:55.139624 (+ 1069us) negotiation.cc:107] Waiting for socket to connect
0806 18:35:55.139635 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0806 18:35:55.139701 (+ 66us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:35:55.140561 (+ 860us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:35:55.140570 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:35:55.140596 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 18:35:55.140741 (+ 145us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:35:55.140748 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:35:55.141859 (+ 1111us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:35:55.141866 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:35:55.144103 (+ 2237us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:35:55.144116 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:35:55.145408 (+ 1292us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:35:55.145411 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:35:55.145554 (+ 143us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:35:55.145589 (+ 35us) client_negotiation.cc:769] Sending connection context
0806 18:35:55.145689 (+ 100us) client_negotiation.cc:240] Negotiation successful
0806 18:35:55.145792 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1019,"thread_start_us":82,"threads_started":1}
I20260806 18:35:55.147260 32381 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:35:55.138966 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59036 (local address 127.19.194.2:40377)
0806 18:35:55.139855 (+ 889us) server_negotiation.cc:206] Beginning negotiation
0806 18:35:55.139864 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0806 18:35:55.139911 (+ 47us) server_negotiation.cc:407] Connection header received
0806 18:35:55.140000 (+ 89us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:35:55.140008 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:35:55.140222 (+ 214us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:35:55.140376 (+ 154us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 18:35:55.140932 (+ 556us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:35:55.141747 (+ 815us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:35:55.144263 (+ 2516us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:35:55.145093 (+ 830us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:35:55.145213 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:35:55.145350 (+ 137us) server_negotiation.cc:1035] Waiting for connection context
0806 18:35:55.146965 (+ 1615us) server_negotiation.cc:299] Negotiation successful
0806 18:35:55.147074 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":801,"thread_start_us":117,"threads_started":1}
I20260806 18:35:55.149085 32377 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:35:55.136760 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.194.3:44499 (local address 127.0.0.1:32946)
0806 18:35:55.139057 (+ 2297us) negotiation.cc:107] Waiting for socket to connect
0806 18:35:55.139072 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0806 18:35:55.139186 (+ 114us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:35:55.142511 (+ 3325us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:35:55.142514 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:35:55.142531 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 18:35:55.142609 (+ 78us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:35:55.142614 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:35:55.144811 (+ 2197us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:35:55.144814 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:35:55.145945 (+ 1131us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:35:55.145958 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:35:55.148726 (+ 2768us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:35:55.148729 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:35:55.148814 (+ 85us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:35:55.148837 (+ 23us) client_negotiation.cc:769] Sending connection context
0806 18:35:55.148883 (+ 46us) client_negotiation.cc:240] Negotiation successful
0806 18:35:55.148953 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2236,"mutex_wait_us":2,"thread_start_us":83,"threads_started":1}
I20260806 18:35:55.149730 32379 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:35:55.137579 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32946 (local address 127.19.194.3:44499)
0806 18:35:55.142057 (+ 4478us) server_negotiation.cc:206] Beginning negotiation
0806 18:35:55.142068 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0806 18:35:55.142088 (+ 20us) server_negotiation.cc:407] Connection header received
0806 18:35:55.142158 (+ 70us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:35:55.142163 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:35:55.142328 (+ 165us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:35:55.142750 (+ 422us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 18:35:55.142879 (+ 129us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:35:55.144070 (+ 1191us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:35:55.146131 (+ 2061us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:35:55.146895 (+ 764us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:35:55.149332 (+ 2437us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:35:55.149466 (+ 134us) server_negotiation.cc:1035] Waiting for connection context
0806 18:35:55.149513 (+ 47us) server_negotiation.cc:299] Negotiation successful
0806 18:35:55.149593 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":4380,"thread_start_us":125,"threads_started":1}
I20260806 18:35:55.150784 32143 tablet_service.cc:1467] Processing CreateTablet for tablet 12f766a91df3410cb6ea0d396bf79ae9 (DEFAULT_TABLE table=test-table [id=1ff5881aaae24d96854501a2be2df94f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 18:35:55.151983 32143 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 12f766a91df3410cb6ea0d396bf79ae9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 18:35:55.152360 32292 tablet_service.cc:1467] Processing CreateTablet for tablet 12f766a91df3410cb6ea0d396bf79ae9 (DEFAULT_TABLE table=test-table [id=1ff5881aaae24d96854501a2be2df94f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 18:35:55.152863 32376 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:35:55.136752 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.194.1:42923 (local address 127.0.0.1:59338)
0806 18:35:55.138661 (+ 1909us) negotiation.cc:107] Waiting for socket to connect
0806 18:35:55.138708 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0806 18:35:55.138828 (+ 120us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:35:55.147046 (+ 8218us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:35:55.147049 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:35:55.147067 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 18:35:55.147189 (+ 122us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:35:55.147195 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:35:55.148633 (+ 1438us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:35:55.148639 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:35:55.149741 (+ 1102us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:35:55.149754 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:35:55.152495 (+ 2741us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:35:55.152499 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:35:55.152587 (+ 88us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:35:55.152610 (+ 23us) client_negotiation.cc:769] Sending connection context
0806 18:35:55.152658 (+ 48us) client_negotiation.cc:240] Negotiation successful
0806 18:35:55.152718 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1733,"thread_start_us":85,"threads_started":1}
I20260806 18:35:55.153328 32292 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 12f766a91df3410cb6ea0d396bf79ae9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 18:35:55.153822 32378 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:35:55.137629 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59338 (local address 127.19.194.1:42923)
0806 18:35:55.144331 (+ 6702us) server_negotiation.cc:206] Beginning negotiation
0806 18:35:55.144339 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 18:35:55.144363 (+ 24us) server_negotiation.cc:407] Connection header received
0806 18:35:55.144429 (+ 66us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:35:55.144494 (+ 65us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:35:55.144665 (+ 171us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:35:55.147337 (+ 2672us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 18:35:55.147480 (+ 143us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:35:55.148522 (+ 1042us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:35:55.151251 (+ 2729us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:35:55.152337 (+ 1086us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:35:55.153320 (+ 983us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:35:55.153537 (+ 217us) server_negotiation.cc:1035] Waiting for connection context
0806 18:35:55.153599 (+ 62us) server_negotiation.cc:299] Negotiation successful
0806 18:35:55.153687 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":6594,"thread_start_us":73,"threads_started":1}
I20260806 18:35:55.156279 32001 tablet_service.cc:1467] Processing CreateTablet for tablet 12f766a91df3410cb6ea0d396bf79ae9 (DEFAULT_TABLE table=test-table [id=1ff5881aaae24d96854501a2be2df94f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 18:35:55.157220 32001 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 12f766a91df3410cb6ea0d396bf79ae9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 18:35:55.163398 32382 tablet_bootstrap.cc:492] T 12f766a91df3410cb6ea0d396bf79ae9 P 838e3fc117634b71b4073ada1f1524d1: Bootstrap starting.
I20260806 18:35:55.166476 32382 tablet_bootstrap.cc:654] T 12f766a91df3410cb6ea0d396bf79ae9 P 838e3fc117634b71b4073ada1f1524d1: Neither blocks nor log segments found. Creating new log.
I20260806 18:35:55.167840 32382 log.cc:824] T 12f766a91df3410cb6ea0d396bf79ae9 P 838e3fc117634b71b4073ada1f1524d1: Log is configured to *not* fsync() on all Append() calls
I20260806 18:35:55.168598 32383 tablet_bootstrap.cc:492] T 12f766a91df3410cb6ea0d396bf79ae9 P 95d8c1822545422193c5994cc716f6e7: Bootstrap starting.
I20260806 18:35:55.169970 32382 tablet_bootstrap.cc:492] T 12f766a91df3410cb6ea0d396bf79ae9 P 838e3fc117634b71b4073ada1f1524d1: No bootstrap required, opened a new log
I20260806 18:35:55.170167 32382 ts_tablet_manager.cc:1397] T 12f766a91df3410cb6ea0d396bf79ae9 P 838e3fc117634b71b4073ada1f1524d1: Time spent bootstrapping tablet: real 0.007s user 0.004s sys 0.000s
I20260806 18:35:55.171414 32383 tablet_bootstrap.cc:654] T 12f766a91df3410cb6ea0d396bf79ae9 P 95d8c1822545422193c5994cc716f6e7: Neither blocks nor log segments found. Creating new log.
I20260806 18:35:55.172261 32383 log.cc:824] T 12f766a91df3410cb6ea0d396bf79ae9 P 95d8c1822545422193c5994cc716f6e7: Log is configured to *not* fsync() on all Append() calls
I20260806 18:35:55.172660 32384 tablet_bootstrap.cc:492] T 12f766a91df3410cb6ea0d396bf79ae9 P 01b2b8b4f9d7433490cddb51513a986f: Bootstrap starting.
I20260806 18:35:55.175190 32384 tablet_bootstrap.cc:654] T 12f766a91df3410cb6ea0d396bf79ae9 P 01b2b8b4f9d7433490cddb51513a986f: Neither blocks nor log segments found. Creating new log.
I20260806 18:35:55.175302 32382 raft_consensus.cc:348] T 12f766a91df3410cb6ea0d396bf79ae9 P 838e3fc117634b71b4073ada1f1524d1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01b2b8b4f9d7433490cddb51513a986f" member_type: VOTER last_known_addr { host: "127.19.194.1" port: 42923 } } peers { permanent_uuid: "95d8c1822545422193c5994cc716f6e7" member_type: VOTER last_known_addr { host: "127.19.194.3" port: 44499 } } peers { permanent_uuid: "838e3fc117634b71b4073ada1f1524d1" member_type: VOTER last_known_addr { host: "127.19.194.2" port: 40377 } }
I20260806 18:35:55.175594 32382 raft_consensus.cc:374] T 12f766a91df3410cb6ea0d396bf79ae9 P 838e3fc117634b71b4073ada1f1524d1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 18:35:55.175649 32382 raft_consensus.cc:729] T 12f766a91df3410cb6ea0d396bf79ae9 P 838e3fc117634b71b4073ada1f1524d1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 838e3fc117634b71b4073ada1f1524d1, State: Initialized, Role: FOLLOWER
I20260806 18:35:55.176100 32382 consensus_queue.cc:260] T 12f766a91df3410cb6ea0d396bf79ae9 P 838e3fc117634b71b4073ada1f1524d1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01b2b8b4f9d7433490cddb51513a986f" member_type: VOTER last_known_addr { host: "127.19.194.1" port: 42923 } } peers { permanent_uuid: "95d8c1822545422193c5994cc716f6e7" member_type: VOTER last_known_addr { host: "127.19.194.3" port: 44499 } } peers { permanent_uuid: "838e3fc117634b71b4073ada1f1524d1" member_type: VOTER last_known_addr { host: "127.19.194.2" port: 40377 } }
I20260806 18:35:55.176533 32384 log.cc:824] T 12f766a91df3410cb6ea0d396bf79ae9 P 01b2b8b4f9d7433490cddb51513a986f: Log is configured to *not* fsync() on all Append() calls
I20260806 18:35:55.177117 32382 ts_tablet_manager.cc:1428] T 12f766a91df3410cb6ea0d396bf79ae9 P 838e3fc117634b71b4073ada1f1524d1: Time spent starting tablet: real 0.007s user 0.004s sys 0.003s
I20260806 18:35:55.177476 32216 heartbeater.cc:503] Master 127.19.194.62:41165 was elected leader, sending a full tablet report...
I20260806 18:35:55.178499 32383 tablet_bootstrap.cc:492] T 12f766a91df3410cb6ea0d396bf79ae9 P 95d8c1822545422193c5994cc716f6e7: No bootstrap required, opened a new log
I20260806 18:35:55.178671 32383 ts_tablet_manager.cc:1397] T 12f766a91df3410cb6ea0d396bf79ae9 P 95d8c1822545422193c5994cc716f6e7: Time spent bootstrapping tablet: real 0.010s user 0.004s sys 0.000s
I20260806 18:35:55.181864 32383 raft_consensus.cc:348] T 12f766a91df3410cb6ea0d396bf79ae9 P 95d8c1822545422193c5994cc716f6e7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01b2b8b4f9d7433490cddb51513a986f" member_type: VOTER last_known_addr { host: "127.19.194.1" port: 42923 } } peers { permanent_uuid: "95d8c1822545422193c5994cc716f6e7" member_type: VOTER last_known_addr { host: "127.19.194.3" port: 44499 } } peers { permanent_uuid: "838e3fc117634b71b4073ada1f1524d1" member_type: VOTER last_known_addr { host: "127.19.194.2" port: 40377 } }
I20260806 18:35:55.182085 32383 raft_consensus.cc:374] T 12f766a91df3410cb6ea0d396bf79ae9 P 95d8c1822545422193c5994cc716f6e7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 18:35:55.182154 32383 raft_consensus.cc:729] T 12f766a91df3410cb6ea0d396bf79ae9 P 95d8c1822545422193c5994cc716f6e7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 95d8c1822545422193c5994cc716f6e7, State: Initialized, Role: FOLLOWER
I20260806 18:35:55.182605 32383 consensus_queue.cc:260] T 12f766a91df3410cb6ea0d396bf79ae9 P 95d8c1822545422193c5994cc716f6e7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01b2b8b4f9d7433490cddb51513a986f" member_type: VOTER last_known_addr { host: "127.19.194.1" port: 42923 } } peers { permanent_uuid: "95d8c1822545422193c5994cc716f6e7" member_type: VOTER last_known_addr { host: "127.19.194.3" port: 44499 } } peers { permanent_uuid: "838e3fc117634b71b4073ada1f1524d1" member_type: VOTER last_known_addr { host: "127.19.194.2" port: 40377 } }
I20260806 18:35:55.183491 32383 ts_tablet_manager.cc:1428] T 12f766a91df3410cb6ea0d396bf79ae9 P 95d8c1822545422193c5994cc716f6e7: Time spent starting tablet: real 0.005s user 0.006s sys 0.000s
I20260806 18:35:55.183861 32363 heartbeater.cc:503] Master 127.19.194.62:41165 was elected leader, sending a full tablet report...
I20260806 18:35:55.187500 32384 tablet_bootstrap.cc:492] T 12f766a91df3410cb6ea0d396bf79ae9 P 01b2b8b4f9d7433490cddb51513a986f: No bootstrap required, opened a new log
I20260806 18:35:55.187696 32384 ts_tablet_manager.cc:1397] T 12f766a91df3410cb6ea0d396bf79ae9 P 01b2b8b4f9d7433490cddb51513a986f: Time spent bootstrapping tablet: real 0.015s user 0.002s sys 0.003s
I20260806 18:35:55.190618 32384 raft_consensus.cc:348] T 12f766a91df3410cb6ea0d396bf79ae9 P 01b2b8b4f9d7433490cddb51513a986f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01b2b8b4f9d7433490cddb51513a986f" member_type: VOTER last_known_addr { host: "127.19.194.1" port: 42923 } } peers { permanent_uuid: "95d8c1822545422193c5994cc716f6e7" member_type: VOTER last_known_addr { host: "127.19.194.3" port: 44499 } } peers { permanent_uuid: "838e3fc117634b71b4073ada1f1524d1" member_type: VOTER last_known_addr { host: "127.19.194.2" port: 40377 } }
I20260806 18:35:55.190836 32384 raft_consensus.cc:374] T 12f766a91df3410cb6ea0d396bf79ae9 P 01b2b8b4f9d7433490cddb51513a986f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 18:35:55.190896 32384 raft_consensus.cc:729] T 12f766a91df3410cb6ea0d396bf79ae9 P 01b2b8b4f9d7433490cddb51513a986f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 01b2b8b4f9d7433490cddb51513a986f, State: Initialized, Role: FOLLOWER
I20260806 18:35:55.191355 32384 consensus_queue.cc:260] T 12f766a91df3410cb6ea0d396bf79ae9 P 01b2b8b4f9d7433490cddb51513a986f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01b2b8b4f9d7433490cddb51513a986f" member_type: VOTER last_known_addr { host: "127.19.194.1" port: 42923 } } peers { permanent_uuid: "95d8c1822545422193c5994cc716f6e7" member_type: VOTER last_known_addr { host: "127.19.194.3" port: 44499 } } peers { permanent_uuid: "838e3fc117634b71b4073ada1f1524d1" member_type: VOTER last_known_addr { host: "127.19.194.2" port: 40377 } }
I20260806 18:35:55.191968 32069 heartbeater.cc:503] Master 127.19.194.62:41165 was elected leader, sending a full tablet report...
I20260806 18:35:55.192296 32384 ts_tablet_manager.cc:1428] T 12f766a91df3410cb6ea0d396bf79ae9 P 01b2b8b4f9d7433490cddb51513a986f: Time spent starting tablet: real 0.005s user 0.003s sys 0.000s
W20260806 18:35:55.223080 32217 tablet.cc:2369] T 12f766a91df3410cb6ea0d396bf79ae9 P 838e3fc117634b71b4073ada1f1524d1: Can't schedule compaction. Clean time has not been advanced past its initial value.
Aug 06 18:35:55 dist-test-slave-1xrh krb5kdc[31351](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041354, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.194.62@KRBTEST.COM for HTTP/127.19.194.20@KRBTEST.COM
I20260806 18:35:55.255332 32390 raft_consensus.cc:482] T 12f766a91df3410cb6ea0d396bf79ae9 P 95d8c1822545422193c5994cc716f6e7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 18:35:55.255565 32390 raft_consensus.cc:504] T 12f766a91df3410cb6ea0d396bf79ae9 P 95d8c1822545422193c5994cc716f6e7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01b2b8b4f9d7433490cddb51513a986f" member_type: VOTER last_known_addr { host: "127.19.194.1" port: 42923 } } peers { permanent_uuid: "95d8c1822545422193c5994cc716f6e7" member_type: VOTER last_known_addr { host: "127.19.194.3" port: 44499 } } peers { permanent_uuid: "838e3fc117634b71b4073ada1f1524d1" member_type: VOTER last_known_addr { host: "127.19.194.2" port: 40377 } }
I20260806 18:35:55.257009 32390 leader_election.cc:288] T 12f766a91df3410cb6ea0d396bf79ae9 P 95d8c1822545422193c5994cc716f6e7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 01b2b8b4f9d7433490cddb51513a986f (127.19.194.1:42923), 838e3fc117634b71b4073ada1f1524d1 (127.19.194.2:40377)
I20260806 18:35:55.264932 32365 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:35:55.257366 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.194.2:40377 (local address 127.19.194.3:60717)
0806 18:35:55.260329 (+ 2963us) negotiation.cc:107] Waiting for socket to connect
0806 18:35:55.260344 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0806 18:35:55.260441 (+ 97us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:35:55.260818 (+ 377us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:35:55.260822 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:35:55.260901 (+ 79us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 18:35:55.261023 (+ 122us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:35:55.261030 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:35:55.262130 (+ 1100us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:35:55.262134 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:35:55.263420 (+ 1286us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:35:55.263431 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:35:55.264497 (+ 1066us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:35:55.264501 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:35:55.264633 (+ 132us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:35:55.264656 (+ 23us) client_negotiation.cc:769] Sending connection context
0806 18:35:55.264726 (+ 70us) client_negotiation.cc:240] Negotiation successful
0806 18:35:55.264795 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2875,"mutex_wait_us":22,"spinlock_wait_cycles":19328}
I20260806 18:35:55.265472 32381 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:35:55.260344 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.194.3:60717 (local address 127.19.194.2:40377)
0806 18:35:55.260483 (+ 139us) server_negotiation.cc:206] Beginning negotiation
0806 18:35:55.260487 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 18:35:55.260504 (+ 17us) server_negotiation.cc:407] Connection header received
0806 18:35:55.260560 (+ 56us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:35:55.260563 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:35:55.260613 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:35:55.260691 (+ 78us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 18:35:55.261178 (+ 487us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:35:55.261976 (+ 798us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:35:55.263589 (+ 1613us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:35:55.264261 (+ 672us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:35:55.264382 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:35:55.264437 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0806 18:35:55.265267 (+ 830us) server_negotiation.cc:299] Negotiation successful
0806 18:35:55.265355 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260806 18:35:55.266099 32378 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:35:55.258116 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.194.3:39743 (local address 127.19.194.1:42923)
0806 18:35:55.258252 (+ 136us) server_negotiation.cc:206] Beginning negotiation
0806 18:35:55.258257 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 18:35:55.260771 (+ 2514us) server_negotiation.cc:407] Connection header received
0806 18:35:55.260949 (+ 178us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:35:55.260952 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:35:55.261002 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:35:55.261060 (+ 58us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 18:35:55.261487 (+ 427us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:35:55.262233 (+ 746us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:35:55.263703 (+ 1470us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:35:55.264316 (+ 613us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:35:55.264417 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:35:55.264468 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0806 18:35:55.265887 (+ 1419us) server_negotiation.cc:299] Negotiation successful
0806 18:35:55.265964 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260806 18:35:55.266053 32165 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "12f766a91df3410cb6ea0d396bf79ae9" candidate_uuid: "95d8c1822545422193c5994cc716f6e7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "838e3fc117634b71b4073ada1f1524d1" is_pre_election: true
I20260806 18:35:55.266350 32165 raft_consensus.cc:2454] T 12f766a91df3410cb6ea0d396bf79ae9 P 838e3fc117634b71b4073ada1f1524d1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 95d8c1822545422193c5994cc716f6e7 in term 0.
I20260806 18:35:55.266530 32393 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:35:55.257668 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.194.1:42923 (local address 127.19.194.3:39743)
0806 18:35:55.260685 (+ 3017us) negotiation.cc:107] Waiting for socket to connect
0806 18:35:55.260698 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0806 18:35:55.260840 (+ 142us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:35:55.261219 (+ 379us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:35:55.261222 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:35:55.261239 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 18:35:55.261353 (+ 114us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:35:55.261359 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:35:55.262409 (+ 1050us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:35:55.262412 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:35:55.263554 (+ 1142us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:35:55.263567 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:35:55.264574 (+ 1007us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:35:55.264578 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:35:55.265720 (+ 1142us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:35:55.265747 (+ 27us) client_negotiation.cc:769] Sending connection context
0806 18:35:55.266299 (+ 552us) client_negotiation.cc:240] Negotiation successful
0806 18:35:55.266372 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2956,"mutex_wait_us":13,"thread_start_us":119,"threads_started":1}
I20260806 18:35:55.267145 32022 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "12f766a91df3410cb6ea0d396bf79ae9" candidate_uuid: "95d8c1822545422193c5994cc716f6e7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "01b2b8b4f9d7433490cddb51513a986f" is_pre_election: true
I20260806 18:35:55.267207 32240 leader_election.cc:302] T 12f766a91df3410cb6ea0d396bf79ae9 P 95d8c1822545422193c5994cc716f6e7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 838e3fc117634b71b4073ada1f1524d1, 95d8c1822545422193c5994cc716f6e7; no voters:
I20260806 18:35:55.267434 32022 raft_consensus.cc:2454] T 12f766a91df3410cb6ea0d396bf79ae9 P 01b2b8b4f9d7433490cddb51513a986f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 95d8c1822545422193c5994cc716f6e7 in term 0.
I20260806 18:35:55.267531 32390 raft_consensus.cc:2781] T 12f766a91df3410cb6ea0d396bf79ae9 P 95d8c1822545422193c5994cc716f6e7 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 18:35:55.267602 32390 raft_consensus.cc:482] T 12f766a91df3410cb6ea0d396bf79ae9 P 95d8c1822545422193c5994cc716f6e7 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 18:35:55.267643 32390 raft_consensus.cc:3037] T 12f766a91df3410cb6ea0d396bf79ae9 P 95d8c1822545422193c5994cc716f6e7 [term 0 FOLLOWER]: Advancing to term 1
I20260806 18:35:55.268965 32390 raft_consensus.cc:504] T 12f766a91df3410cb6ea0d396bf79ae9 P 95d8c1822545422193c5994cc716f6e7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01b2b8b4f9d7433490cddb51513a986f" member_type: VOTER last_known_addr { host: "127.19.194.1" port: 42923 } } peers { permanent_uuid: "95d8c1822545422193c5994cc716f6e7" member_type: VOTER last_known_addr { host: "127.19.194.3" port: 44499 } } peers { permanent_uuid: "838e3fc117634b71b4073ada1f1524d1" member_type: VOTER last_known_addr { host: "127.19.194.2" port: 40377 } }
I20260806 18:35:55.269578 32165 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "12f766a91df3410cb6ea0d396bf79ae9" candidate_uuid: "95d8c1822545422193c5994cc716f6e7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "838e3fc117634b71b4073ada1f1524d1"
I20260806 18:35:55.269711 32165 raft_consensus.cc:3037] T 12f766a91df3410cb6ea0d396bf79ae9 P 838e3fc117634b71b4073ada1f1524d1 [term 0 FOLLOWER]: Advancing to term 1
I20260806 18:35:55.269819 32022 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "12f766a91df3410cb6ea0d396bf79ae9" candidate_uuid: "95d8c1822545422193c5994cc716f6e7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "01b2b8b4f9d7433490cddb51513a986f"
I20260806 18:35:55.269939 32022 raft_consensus.cc:3037] T 12f766a91df3410cb6ea0d396bf79ae9 P 01b2b8b4f9d7433490cddb51513a986f [term 0 FOLLOWER]: Advancing to term 1
I20260806 18:35:55.271106 32022 raft_consensus.cc:2454] T 12f766a91df3410cb6ea0d396bf79ae9 P 01b2b8b4f9d7433490cddb51513a986f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 95d8c1822545422193c5994cc716f6e7 in term 1.
I20260806 18:35:55.271170 32165 raft_consensus.cc:2454] T 12f766a91df3410cb6ea0d396bf79ae9 P 838e3fc117634b71b4073ada1f1524d1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 95d8c1822545422193c5994cc716f6e7 in term 1.
I20260806 18:35:55.271543 32240 leader_election.cc:302] T 12f766a91df3410cb6ea0d396bf79ae9 P 95d8c1822545422193c5994cc716f6e7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 838e3fc117634b71b4073ada1f1524d1, 95d8c1822545422193c5994cc716f6e7; no voters:
I20260806 18:35:55.271716 32390 leader_election.cc:288] T 12f766a91df3410cb6ea0d396bf79ae9 P 95d8c1822545422193c5994cc716f6e7 [CANDIDATE]: Term 1 election: Requested vote from peers 01b2b8b4f9d7433490cddb51513a986f (127.19.194.1:42923), 838e3fc117634b71b4073ada1f1524d1 (127.19.194.2:40377)
I20260806 18:35:55.271911 32394 raft_consensus.cc:2781] T 12f766a91df3410cb6ea0d396bf79ae9 P 95d8c1822545422193c5994cc716f6e7 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 18:35:55.272050 32394 raft_consensus.cc:686] T 12f766a91df3410cb6ea0d396bf79ae9 P 95d8c1822545422193c5994cc716f6e7 [term 1 LEADER]: Becoming Leader. State: Replica: 95d8c1822545422193c5994cc716f6e7, State: Running, Role: LEADER
I20260806 18:35:55.272410 32394 consensus_queue.cc:237] T 12f766a91df3410cb6ea0d396bf79ae9 P 95d8c1822545422193c5994cc716f6e7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01b2b8b4f9d7433490cddb51513a986f" member_type: VOTER last_known_addr { host: "127.19.194.1" port: 42923 } } peers { permanent_uuid: "95d8c1822545422193c5994cc716f6e7" member_type: VOTER last_known_addr { host: "127.19.194.3" port: 44499 } } peers { permanent_uuid: "838e3fc117634b71b4073ada1f1524d1" member_type: VOTER last_known_addr { host: "127.19.194.2" port: 40377 } }
I20260806 18:35:55.276273 31849 catalog_manager.cc:5495] T 12f766a91df3410cb6ea0d396bf79ae9 P 95d8c1822545422193c5994cc716f6e7 reported cstate change: term changed from 0 to 1, leader changed from <none> to 95d8c1822545422193c5994cc716f6e7 (127.19.194.3). New cstate: current_term: 1 leader_uuid: "95d8c1822545422193c5994cc716f6e7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01b2b8b4f9d7433490cddb51513a986f" member_type: VOTER last_known_addr { host: "127.19.194.1" port: 42923 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "95d8c1822545422193c5994cc716f6e7" member_type: VOTER last_known_addr { host: "127.19.194.3" port: 44499 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "838e3fc117634b71b4073ada1f1524d1" member_type: VOTER last_known_addr { host: "127.19.194.2" port: 40377 } health_report { overall_health: UNKNOWN } } }
I20260806 18:35:55.308346 32379 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:35:55.303247 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32952 (local address 127.19.194.3:44499)
0806 18:35:55.303515 (+ 268us) server_negotiation.cc:206] Beginning negotiation
0806 18:35:55.303520 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 18:35:55.303538 (+ 18us) server_negotiation.cc:407] Connection header received
0806 18:35:55.303620 (+ 82us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:35:55.303623 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:35:55.303678 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:35:55.303810 (+ 132us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 18:35:55.304092 (+ 282us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:35:55.304613 (+ 521us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:35:55.305992 (+ 1379us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:35:55.306390 (+ 398us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:35:55.306496 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:35:55.306917 (+ 421us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 18:35:55.307486 (+ 569us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 18:35:55.307614 (+ 128us) server_negotiation.cc:1035] Waiting for connection context
0806 18:35:55.308126 (+ 512us) server_negotiation.cc:299] Negotiation successful
0806 18:35:55.308191 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":184}
I20260806 18:35:55.321673 32165 raft_consensus.cc:1261] T 12f766a91df3410cb6ea0d396bf79ae9 P 838e3fc117634b71b4073ada1f1524d1 [term 1 FOLLOWER]: Refusing update from remote peer 95d8c1822545422193c5994cc716f6e7: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 18:35:55.322500 32394 consensus_queue.cc:1035] T 12f766a91df3410cb6ea0d396bf79ae9 P 95d8c1822545422193c5994cc716f6e7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "838e3fc117634b71b4073ada1f1524d1" member_type: VOTER last_known_addr { host: "127.19.194.2" port: 40377 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 18:35:55.330236 32022 raft_consensus.cc:1261] T 12f766a91df3410cb6ea0d396bf79ae9 P 01b2b8b4f9d7433490cddb51513a986f [term 1 FOLLOWER]: Refusing update from remote peer 95d8c1822545422193c5994cc716f6e7: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 18:35:55.330940 32394 consensus_queue.cc:1035] T 12f766a91df3410cb6ea0d396bf79ae9 P 95d8c1822545422193c5994cc716f6e7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "01b2b8b4f9d7433490cddb51513a986f" member_type: VOTER last_known_addr { host: "127.19.194.1" port: 42923 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 18:35:55.336578 32398 mvcc.cc:205] Tried to move back new op lower bound from 7315625391387959296 to 7315625391198752768. Current Snapshot: MvccSnapshot[applied={T|T < 7315625391387959296}]
I20260806 18:35:55.341717 32400 mvcc.cc:205] Tried to move back new op lower bound from 7315625391387959296 to 7315625391198752768. Current Snapshot: MvccSnapshot[applied={T|T < 7315625391387959296}]
I20260806 18:35:55.344115 32402 mvcc.cc:205] Tried to move back new op lower bound from 7315625391387959296 to 7315625391198752768. Current Snapshot: MvccSnapshot[applied={T|T < 7315625391387959296}]
I20260806 18:35:55.353425 31848 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:42602:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 18:35:55.353605 31848 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:42602:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 18:35:55.361313 31848 catalog_manager.cc:5782] T 00000000000000000000000000000000 P e05daed5b6f74deeabdfff19b89c06ee: Sending DeleteTablet for 3 replicas of tablet 12f766a91df3410cb6ea0d396bf79ae9
I20260806 18:35:55.362087 32001 tablet_service.cc:1514] Processing DeleteTablet for tablet 12f766a91df3410cb6ea0d396bf79ae9 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 18:35:55 UTC) from {username='kudu', principal='kudu/127.19.194.62@KRBTEST.COM'} at 127.0.0.1:59338
I20260806 18:35:55.363095 32292 tablet_service.cc:1514] Processing DeleteTablet for tablet 12f766a91df3410cb6ea0d396bf79ae9 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 18:35:55 UTC) from {username='kudu', principal='kudu/127.19.194.62@KRBTEST.COM'} at 127.0.0.1:32946
I20260806 18:35:55.364068 32143 tablet_service.cc:1514] Processing DeleteTablet for tablet 12f766a91df3410cb6ea0d396bf79ae9 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 18:35:55 UTC) from {username='kudu', principal='kudu/127.19.194.62@KRBTEST.COM'} at 127.0.0.1:59036
I20260806 18:35:55.365525 20232 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJW69tn/build/debug/bin/kudu with pid 31934
I20260806 18:35:55.376860 32411 tablet_replica.cc:330] T 12f766a91df3410cb6ea0d396bf79ae9 P 838e3fc117634b71b4073ada1f1524d1: stopping tablet replica
I20260806 18:35:55.377287 32411 raft_consensus.cc:2229] T 12f766a91df3410cb6ea0d396bf79ae9 P 838e3fc117634b71b4073ada1f1524d1 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 18:35:55.377884 32410 tablet_replica.cc:330] T 12f766a91df3410cb6ea0d396bf79ae9 P 95d8c1822545422193c5994cc716f6e7: stopping tablet replica
I20260806 18:35:55.378198 32410 raft_consensus.cc:2229] T 12f766a91df3410cb6ea0d396bf79ae9 P 95d8c1822545422193c5994cc716f6e7 [term 1 LEADER]: Raft consensus shutting down.
I20260806 18:35:55.378654 32410 raft_consensus.cc:2258] T 12f766a91df3410cb6ea0d396bf79ae9 P 95d8c1822545422193c5994cc716f6e7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 18:35:55.379182 32411 raft_consensus.cc:2258] T 12f766a91df3410cb6ea0d396bf79ae9 P 838e3fc117634b71b4073ada1f1524d1 [term 1 FOLLOWER]: Raft consensus is shut down!
W20260806 18:35:55.383776 31836 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.19.194.1:42923 (error 108)
I20260806 18:35:55.388531 32411 ts_tablet_manager.cc:1905] T 12f766a91df3410cb6ea0d396bf79ae9 P 838e3fc117634b71b4073ada1f1524d1: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 18:35:55.388842 20232 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJW69tn/build/debug/bin/kudu with pid 32077
I20260806 18:35:55.393117 32410 ts_tablet_manager.cc:1905] T 12f766a91df3410cb6ea0d396bf79ae9 P 95d8c1822545422193c5994cc716f6e7: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 18:35:55.396394 32410 ts_tablet_manager.cc:1918] T 12f766a91df3410cb6ea0d396bf79ae9 P 95d8c1822545422193c5994cc716f6e7: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 18:35:55.396497 32410 log.cc:1196] T 12f766a91df3410cb6ea0d396bf79ae9 P 95d8c1822545422193c5994cc716f6e7: Deleting WAL directory at /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786041246094603-20232-0/minicluster-data/ts-2/wal/wals/12f766a91df3410cb6ea0d396bf79ae9
I20260806 18:35:55.396847 32410 ts_tablet_manager.cc:1939] T 12f766a91df3410cb6ea0d396bf79ae9 P 95d8c1822545422193c5994cc716f6e7: Deleting consensus metadata
I20260806 18:35:55.400106 32376 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:35:55.399741 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.194.1:42923 (local address 127.0.0.1:59344)
0806 18:35:55.399941 (+ 200us) negotiation.cc:107] Waiting for socket to connect
0806 18:35:55.400012 (+ 71us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.194.1:42923: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":120}
W20260806 18:35:55.401592 31836 catalog_manager.cc:4584] TS 01b2b8b4f9d7433490cddb51513a986f (127.19.194.1:42923): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 12f766a91df3410cb6ea0d396bf79ae9: Network error: Client connection negotiation failed: client connection to 127.19.194.1:42923: connect: Connection refused (error 111)
I20260806 18:35:55.404666 31833 catalog_manager.cc:4841] TS 95d8c1822545422193c5994cc716f6e7 (127.19.194.3:44499): tablet 12f766a91df3410cb6ea0d396bf79ae9 (table test-table [id=1ff5881aaae24d96854501a2be2df94f]) successfully deleted
W20260806 18:35:55.405570 32376 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 18:35:55.404997 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.194.2:40377 (local address 127.0.0.1:59046)
0806 18:35:55.405173 (+ 176us) negotiation.cc:107] Waiting for socket to connect
0806 18:35:55.405187 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0806 18:35:55.405268 (+ 81us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:35:55.405477 (+ 209us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.194.2:40377: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
Metrics: {"client-negotiator.queue_time_us":107}
I20260806 18:35:55.406236 20232 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJW69tn/build/debug/bin/kudu with pid 32221
I20260806 18:35:55.419044 20232 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJW69tn/build/debug/bin/kudu with pid 31815
I20260806 18:35:55.429649 20232 mini_ranger_kms.cc:60] Stopping Ranger KMS...
I20260806 18:35:55.774133 20232 mini_ranger_kms.cc:62] Stopped Ranger KMS
I20260806 18:35:55.774292 20232 mini_ranger.cc:66] Stopping Ranger...
I20260806 18:35:56.118614 20232 mini_ranger.cc:68] Stopped Ranger
2026-08-06 18:35:56.118 UTC [31382] LOG: received smart shutdown request
2026-08-06 18:35:56.122 UTC [31382] LOG: background worker "logical replication launcher" (PID 31391) exited with exit code 1
2026-08-06 18:35:56.123 UTC [31386] LOG: shutting down
2026-08-06 18:35:56.291 UTC [31382] LOG: database system is shut down
2026-08-06T18:35:56Z chronyd exiting
[ OK ] SecurityITest.TestEncryptionWithKMSIntegration (47268 ms)
[ RUN ] SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers
Loading random data
Initializing database '/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786041246094603-20232-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 18:35:56 dist-test-slave-1xrh krb5kdc[32421](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 18:35:56 dist-test-slave-1xrh krb5kdc[32421](info): set up 2 sockets
Aug 06 18:35:56 dist-test-slave-1xrh krb5kdc[32421](info): commencing operation
krb5kdc: starting...
W20260806 18:35:57.522058 20232 mini_kdc.cc:121] Time spent starting KDC: real 1.075s user 0.003s sys 0.003s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786041246094603-20232-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786041246094603-20232-0/krb5kdc/test-user.keytab.
Aug 06 18:35:57 dist-test-slave-1xrh krb5kdc[32421](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041357, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T18:35:57Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T18:35:57Z Disabled control of system clock
WARNING: no policy specified for rangeradmin/127.19.194.20@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangeradmin/127.19.194.20@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangeradmin/127.19.194.20@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786041246094603-20232-0/krb5kdc/rangeradmin_127.19.194.20@KRBTEST.COM.keytab.
Entry for principal rangeradmin/127.19.194.20@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786041246094603-20232-0/krb5kdc/rangeradmin_127.19.194.20@KRBTEST.COM.keytab.
WARNING: no policy specified for rangerlookup/127.19.194.20@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerlookup/127.19.194.20@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerlookup/127.19.194.20@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786041246094603-20232-0/krb5kdc/rangerlookup_127.19.194.20@KRBTEST.COM.keytab.
Entry for principal rangerlookup/127.19.194.20@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786041246094603-20232-0/krb5kdc/rangerlookup_127.19.194.20@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.19.194.20@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "HTTP/127.19.194.20@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.19.194.20@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786041246094603-20232-0/krb5kdc/HTTP_127.19.194.20@KRBTEST.COM.keytab.
Entry for principal HTTP/127.19.194.20@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786041246094603-20232-0/krb5kdc/HTTP_127.19.194.20@KRBTEST.COM.keytab.
I20260806 18:35:57.689162 20232 mini_postgres.cc:62] Running initdb...
The files belonging to this database system will be owned by user "slave".
This user must also own the server process.
The database cluster will be initialized with locale "C".
The default database encoding has accordingly been set to "SQL_ASCII".
The default text search configuration will be set to "english".
Data page checksums are disabled.
creating directory /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786041246094603-20232-0/minicluster-data/postgres ... ok
creating subdirectories ... ok
selecting dynamic shared memory implementation ... posix
selecting default max_connections ... 100
selecting default shared_buffers ... 128MB
selecting default time zone ... Etc/UTC
creating configuration files ... ok
running bootstrap script ... ok
performing post-bootstrap initialization ... ok
syncing data to disk ... ok
initdb: warning: enabling "trust" authentication for local connections
You can change this by editing pg_hba.conf or using the option -A, or
--auth-local and --auth-host, the next time you run initdb.
Success. You can now start the database server using:
/tmp/dist-test-taskJW69tn/build/debug/bin/postgres/pg_ctl -D /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786041246094603-20232-0/minicluster-data/postgres -l logfile start
2026-08-06 18:36:00.148 UTC [32454] LOG: starting PostgreSQL 12.2 on x86_64-pc-linux-gnu, compiled by cc (GCC) 8.3.1 20190311 (Red Hat 8.3.1-3), 64-bit
2026-08-06 18:36:00.148 UTC [32454] LOG: listening on IPv4 address "127.19.194.20", port 51083
2026-08-06 18:36:00.152 UTC [32454] LOG: listening on Unix socket "/tmp/.s.PGSQL.51083"
2026-08-06 18:36:00.172 UTC [32457] LOG: database system was shut down at 2026-08-06 18:35:58 UTC
2026-08-06 18:36:00.176 UTC [32454] LOG: database system is ready to accept connections
I20260806 18:36:01.152510 20232 mini_postgres.cc:96] Postgres bound to 51083
2026-08-06 18:36:01.159 UTC [32466] FATAL: database "slave" does not exist
127.19.194.20:51083 - accepting connections
I20260806 18:36:01.160034 20232 mini_ranger.cc:161] Starting Ranger...
I20260806 18:36:01.175374 20232 mini_ranger.cc:84] Created miniranger Postgres user
I20260806 18:36:01.636217 20232 mini_ranger.cc:87] Created ranger Postgres database
I20260806 18:36:01.636291 20232 mini_ranger.cc:178] Starting Ranger out of /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786041246094603-20232-0/minicluster-data/ranger-admin
2026-08-06 18:36:01,919 [I] DB FLAVOR :POSTGRES
2026-08-06 18:36:01,920 [I] --------- Verifying Ranger DB connection ---------
2026-08-06 18:36:01,920 [I] Checking connection..
2026-08-06 18:36:01,920 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskJW69tn/build/debug/bin/postgresql.jar:/tmp/dist-test-taskJW69tn/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.19.194.20:51083/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select 1;"
2026-08-06 18:36:02,174 [I] Checking connection passed.
2026-08-06 18:36:02,175 [I] --------- Verifying version history table ---------
2026-08-06 18:36:02,175 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskJW69tn/build/debug/bin/postgresql.jar:/tmp/dist-test-taskJW69tn/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.19.194.20:51083/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-06 18:36:02,406 [I] Table x_db_version_h does not exist in database ranger
2026-08-06 18:36:02,406 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskJW69tn/build/debug/bin/postgresql.jar:/tmp/dist-test-taskJW69tn/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.19.194.20:51083/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-06 18:36:02,641 [I] Table x_db_version_h does not exist in database ranger
2026-08-06 18:36:02,642 [I] Importing x_db_version_h table schema to database ranger from file: create_dbversion_catalog.sql
2026-08-06 18:36:02,642 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskJW69tn/build/debug/bin/postgresql.jar:/tmp/dist-test-taskJW69tn/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.19.194.20:51083/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskJW69tn/thirdparty/src/ranger-2.1.0-admin/db/postgres/create_dbversion_catalog.sql
2026-08-06 18:36:02.883 UTC [32535] WARNING: there is no transaction in progress
2026-08-06 18:36:02,890 [I] create_dbversion_catalog.sql file imported successfully
2026-08-06 18:36:02,890 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskJW69tn/build/debug/bin/postgresql.jar:/tmp/dist-test-taskJW69tn/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.19.194.20:51083/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-06 18:36:03,114 [I] Table x_db_version_h already exists in database 'ranger'
2026-08-06 18:36:03,114 [I] --------- Importing Ranger Core DB Schema ---------
2026-08-06 18:36:03,115 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskJW69tn/build/debug/bin/postgresql.jar:/tmp/dist-test-taskJW69tn/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.19.194.20:51083/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'Y';"
2026-08-06 18:36:03,375 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskJW69tn/build/debug/bin/postgresql.jar:/tmp/dist-test-taskJW69tn/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.19.194.20:51083/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'N';"
2026-08-06 18:36:03,648 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskJW69tn/build/debug/bin/postgresql.jar:/tmp/dist-test-taskJW69tn/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.19.194.20:51083/ranger -u miniranger -p '********' -noheader -trim -c \; -query "insert into x_db_version_h (version, inst_at, inst_by, updated_at, updated_by,active) values ('CORE_DB_SCHEMA', current_timestamp, 'Ranger 2.1.0', current_timestamp, 'dist-test-slave-1xrh.c.gcp-upstream.internal','N') ;"
2026-08-06 18:36:03,894 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskJW69tn/build/debug/bin/postgresql.jar:/tmp/dist-test-taskJW69tn/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.19.194.20:51083/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-08-06 18:36:04,155 [I] Table x_portal_user does not exist in database ranger
2026-08-06 18:36:04,156 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskJW69tn/build/debug/bin/postgresql.jar:/tmp/dist-test-taskJW69tn/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.19.194.20:51083/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-08-06 18:36:04,389 [I] Table x_policy_ref_group does not exist in database ranger
2026-08-06 18:36:04,390 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskJW69tn/build/debug/bin/postgresql.jar:/tmp/dist-test-taskJW69tn/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.19.194.20:51083/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-08-06 18:36:04,626 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskJW69tn/build/debug/bin/postgresql.jar:/tmp/dist-test-taskJW69tn/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.19.194.20:51083/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-08-06 18:36:04,879 [I] Importing DB schema to database ranger from file: ranger_core_db_postgres.sql
2026-08-06 18:36:04,880 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskJW69tn/build/debug/bin/postgresql.jar:/tmp/dist-test-taskJW69tn/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.19.194.20:51083/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskJW69tn/thirdparty/src/ranger-2.1.0-admin/db/postgres/optimized/current/ranger_core_db_postgres.sql
2026-08-06 18:36:05.807 UTC [32649] WARNING: there is no transaction in progress
2026-08-06 18:36:05.823 UTC [32649] WARNING: there is no transaction in progress
2026-08-06 18:36:05.839 UTC [32649] WARNING: there is no transaction in progress
2026-08-06 18:36:05.855 UTC [32649] WARNING: there is no transaction in progress
2026-08-06 18:36:05.871 UTC [32649] WARNING: there is no transaction in progress
2026-08-06 18:36:05.887 UTC [32649] WARNING: there is no transaction in progress
2026-08-06 18:36:05.939 UTC [32649] WARNING: there is no transaction in progress
2026-08-06 18:36:05.946 UTC [32649] WARNING: there is no transaction in progress
2026-08-06 18:36:05.961 UTC [32649] WARNING: there is no transaction in progress
2026-08-06 18:36:05.971 UTC [32649] WARNING: there is no transaction in progress
2026-08-06 18:36:05.981 UTC [32649] WARNING: there is no transaction in progress
2026-08-06 18:36:05.995 UTC [32649] WARNING: there is no transaction in progress
2026-08-06 18:36:06.004 UTC [32649] WARNING: there is no transaction in progress
2026-08-06 18:36:06.011 UTC [32649] WARNING: there is no transaction in progress
2026-08-06 18:36:06.017 UTC [32649] WARNING: there is no transaction in progress
2026-08-06 18:36:06.024 UTC [32649] WARNING: there is no transaction in progress
2026-08-06 18:36:06,643 [I] ranger_core_db_postgres.sql file imported successfully
2026-08-06 18:36:06,643 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskJW69tn/build/debug/bin/postgresql.jar:/tmp/dist-test-taskJW69tn/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.19.194.20:51083/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set inst_by='Ranger 2.1.0' where active='Y' and updated_by='localhost';"
2026-08-06 18:36:06,892 [I] Patches status entries updated from base ranger version to current installed ranger version:Ranger 2.1.0
2026-08-06 18:36:06,892 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskJW69tn/build/debug/bin/postgresql.jar:/tmp/dist-test-taskJW69tn/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.19.194.20:51083/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-08-06 18:36:07,149 [I] Table x_portal_user already exists in database 'ranger'
2026-08-06 18:36:07,149 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskJW69tn/build/debug/bin/postgresql.jar:/tmp/dist-test-taskJW69tn/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.19.194.20:51083/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-08-06 18:36:07,403 [I] Table x_policy_ref_group already exists in database 'ranger'
2026-08-06 18:36:07,403 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskJW69tn/build/debug/bin/postgresql.jar:/tmp/dist-test-taskJW69tn/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.19.194.20:51083/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-08-06 18:36:07,650 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskJW69tn/build/debug/bin/postgresql.jar:/tmp/dist-test-taskJW69tn/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.19.194.20:51083/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-08-06 18:36:07,908 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskJW69tn/build/debug/bin/postgresql.jar:/tmp/dist-test-taskJW69tn/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.19.194.20:51083/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set active='Y' where version='CORE_DB_SCHEMA' and active='N' and updated_by='dist-test-slave-1xrh.c.gcp-upstream.internal';"
2026-08-06 18:36:08,123 [I] CORE_DB_SCHEMA import status has been updated
2026-08-06 18:36:08,123 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskJW69tn/build/debug/bin/postgresql.jar:/tmp/dist-test-taskJW69tn/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.19.194.20:51083/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and inst_by = 'Ranger 2.1.0' and active = 'Y';"
2026-08-06 18:36:08,370 [I] DB_PATCHES have already been applied
I20260806 18:36:08.376832 20232 mini_ranger.cc:191] Using Ranger class path: /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786041246094603-20232-0/minicluster-data/ranger-admin:/tmp/dist-test-taskJW69tn/thirdparty/src/ranger-2.1.0-admin/ews/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*:/tmp/dist-test-taskJW69tn/thirdparty/src/hadoop-3.2.0/*:/tmp/dist-test-taskJW69tn/build/debug/bin/postgresql.jar:/tmp/dist-test-taskJW69tn/thirdparty/src/ranger-2.1.0-admin/ews/webapp
I20260806 18:36:08.376894 20232 mini_ranger.cc:193] Using host: 127.19.194.20
I20260806 18:36:08.380936 20232 mini_ranger.cc:227] Ranger admin URL: http://127.19.194.20:46697
Aug 06, 2026 6:36:08 PM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Aug 06, 2026 6:36:08 PM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Aug 06, 2026 6:36:08 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Deriving webapp folder from catalina.base property. folder=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786041246094603-20232-0/minicluster-data/ranger-admin/ews/webapp
Aug 06, 2026 6:36:08 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786041246094603-20232-0/minicluster-data/ranger-admin/ews/webapp, webAppName = /
Aug 06, 2026 6:36:08 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/] = path [/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786041246094603-20232-0/minicluster-data/ranger-admin/ews/webapp] .....
Aug 06, 2026 6:36:09 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/] = path [/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786041246094603-20232-0/minicluster-data/ranger-admin/ews/webapp].
Aug 06, 2026 6:36:09 PM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : miniranger
I20260806 18:36:09.524904 20232 mini_ranger.cc:160] Time spent starting Ranger: real 8.365s user 0.005s sys 0.006s
Aug 06, 2026 6:36:30 PM com.sun.jersey.api.core.PackagesResourceConfig init
INFO: Scanning for root resource and provider classes in the packages:
org.apache.ranger.rest
org.apache.ranger.common
xa.rest
Aug 06, 2026 6:36:30 PM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Root resource classes found:
class org.apache.ranger.rest.MetricsREST
class org.apache.ranger.rest.XKeyREST
class org.apache.ranger.rest.XAuditREST
class org.apache.ranger.rest.XUserREST
class org.apache.ranger.rest.UserREST
class org.apache.ranger.rest.ServiceREST
class org.apache.ranger.rest.PublicAPIs
class org.apache.ranger.rest.TagREST
class org.apache.ranger.rest.AssetREST
class org.apache.ranger.rest.RoleREST
class org.apache.ranger.rest.SecurityZoneREST
class org.apache.ranger.rest.PublicAPIsv2
Aug 06, 2026 6:36:30 PM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Provider classes found:
class org.apache.ranger.common.RangerJAXBContextResolver
Aug 06, 2026 6:36:30 PM com.sun.jersey.spi.spring.container.servlet.SpringServlet getContext
INFO: Using default applicationContext
Aug 06, 2026 6:36:30 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, assetREST, of type org.apache.ranger.rest.AssetREST as a root resource class
Aug 06, 2026 6:36:30 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, metricsREST, of type org.apache.ranger.rest.MetricsREST as a root resource class
Aug 06, 2026 6:36:30 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIs, of type org.apache.ranger.rest.PublicAPIs as a root resource class
Aug 06, 2026 6:36:30 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIsv2, of type org.apache.ranger.rest.PublicAPIsv2 as a root resource class
Aug 06, 2026 6:36:30 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, roleREST, of type org.apache.ranger.rest.RoleREST as a root resource class
Aug 06, 2026 6:36:30 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, securityZoneREST, of type org.apache.ranger.rest.SecurityZoneREST as a root resource class
Aug 06, 2026 6:36:30 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, serviceREST, of type org.apache.ranger.rest.ServiceREST as a root resource class
Aug 06, 2026 6:36:30 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, tagREST, of type org.apache.ranger.rest.TagREST as a root resource class
Aug 06, 2026 6:36:30 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, userREST, of type org.apache.ranger.rest.UserREST as a root resource class
Aug 06, 2026 6:36:30 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XAuditREST, of type org.apache.ranger.rest.XAuditREST as a root resource class
Aug 06, 2026 6:36:30 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XKeyREST, of type org.apache.ranger.rest.XKeyREST as a root resource class
Aug 06, 2026 6:36:30 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XUserREST, of type org.apache.ranger.rest.XUserREST as a root resource class
Aug 06, 2026 6:36:30 PM com.sun.jersey.server.impl.application.WebApplicationImpl _initiate
INFO: Initiating Jersey application, version 'Jersey: 1.19.3 10/24/2016 03:58 PM'
Aug 06, 2026 6:36:31 PM com.sun.jersey.spi.inject.Errors processErrorMessages
WARNING: The following warnings have been detected with resource and/or provider classes:
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInExcel(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInJson(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse,java.lang.Boolean), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInCsv(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse) throws java.io.IOException, MUST return a non-void type.
I20260806 18:36:31.600574 20232 mini_ranger.cc:261] Created Kudu service
WARNING: no policy specified for rangerkms/127.19.194.20@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerkms/127.19.194.20@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerkms/127.19.194.20@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786041246094603-20232-0/krb5kdc/rangerkms_127.19.194.20@KRBTEST.COM.keytab.
Entry for principal rangerkms/127.19.194.20@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786041246094603-20232-0/krb5kdc/rangerkms_127.19.194.20@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.19.194.20@KRBTEST.COM; defaulting to no policy
add_principal: Principal or policy already exists while creating "HTTP/127.19.194.20@KRBTEST.COM".
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.19.194.20@KRBTEST.COM with kvno 3, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786041246094603-20232-0/krb5kdc/HTTP_127.19.194.20@KRBTEST.COM.keytab.
Entry for principal HTTP/127.19.194.20@KRBTEST.COM with kvno 3, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786041246094603-20232-0/krb5kdc/HTTP_127.19.194.20@KRBTEST.COM.keytab.
WARNING: no policy specified for keyadmin@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "keyadmin@KRBTEST.COM" created.
Aug 06 18:36:31 dist-test-slave-1xrh krb5kdc[32421](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041391, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for keyadmin@KRBTEST.COM:
I20260806 18:36:31.685178 20232 mini_ranger_kms.cc:208] Starting Ranger KMS...
I20260806 18:36:31.694147 20232 mini_ranger_kms.cc:78] Created minirangerkms Postgres user
I20260806 18:36:32.553390 20232 mini_ranger_kms.cc:81] Created rangerkms Postgres database
I20260806 18:36:32.553479 20232 mini_ranger_kms.cc:226] Starting Ranger KMS out of /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786041246094603-20232-0/minicluster-data/ranger-kms
I20260806 18:36:32.553498 20232 mini_ranger_kms.cc:227] Using postgres at 127.19.194.20:51083
2026-08-06 18:36:32,734 [I] DB FLAVOR :POSTGRES
2026-08-06 18:36:32,735 [I] --------- Verifying Ranger DB connection ---------
2026-08-06 18:36:32,735 [I] Checking connection
2026-08-06 18:36:32,735 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskJW69tn/build/debug/bin/postgresql.jar:/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786041246094603-20232-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.19.194.20:51083/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "SELECT 1;"
2026-08-06 18:36:32,970 [I] connection success
2026-08-06 18:36:32,970 [I] --------- Verifying Ranger DB tables ---------
2026-08-06 18:36:32,970 [I] Verifying table ranger_masterkey in database rangerkms
2026-08-06 18:36:32,970 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskJW69tn/build/debug/bin/postgresql.jar:/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786041246094603-20232-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.19.194.20:51083/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='rangerkms' and table_name = 'ranger_masterkey') as temp;"
2026-08-06 18:36:33,220 [I] Table ranger_masterkey does not exist in database rangerkms
2026-08-06 18:36:33,220 [I] --------- Importing Ranger Core DB Schema ---------
2026-08-06 18:36:33,220 [I] Importing db schema to database rangerkms from file: kms_core_db_postgres.sql
2026-08-06 18:36:33,221 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskJW69tn/build/debug/bin/postgresql.jar:/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786041246094603-20232-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.19.194.20:51083/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786041246094603-20232-0/minicluster-data/ranger-kms/db/postgres/kms_core_db_postgres.sql
2026-08-06 18:36:33,484 [I] kms_core_db_postgres.sql DB schema imported successfully
I20260806 18:36:33.920656 20232 mini_ranger_kms.cc:310] Created kms service
I20260806 18:36:34.104820 20232 mini_ranger_kms.cc:326] Created kudu user
I20260806 18:36:34.151367 20232 mini_ranger_kms.cc:343] Created rangerkms user
Aug 06 18:36:34 dist-test-slave-1xrh krb5kdc[32421](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041391, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for HTTP/127.19.194.20@KRBTEST.COM
I20260806 18:36:34.543026 20232 mini_ranger_kms.cc:384] Added ranger policy
I20260806 18:36:34.543497 20232 mini_ranger_kms.cc:238] Using RangerKMS classpath: /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786041246094603-20232-0/minicluster-data/ranger-kms:/tmp/dist-test-taskJW69tn/build/debug/bin/postgresql.jar:/tmp/dist-test-taskJW69tn/thirdparty/src/ranger-2.1.0-kms/ews/webapp/WEB-INF/classes/lib/*:/tmp/dist-test-taskJW69tn/thirdparty/src/ranger-2.1.0-kms/ews/webapp/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*
I20260806 18:36:34.543555 20232 mini_ranger_kms.cc:240] Using host: 127.19.194.20
I20260806 18:36:34.547884 20232 mini_ranger_kms.cc:276] Ranger KMS PID: 374
I20260806 18:36:34.547964 20232 mini_ranger_kms.cc:277] Ranger KMS URL: http://127.19.194.20:49915
Aug 06, 2026 6:36:35 PM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Aug 06, 2026 6:36:35 PM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Aug 06, 2026 6:36:35 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786041246094603-20232-0/minicluster-data/ranger-kms/ews/webapp, webAppName = /kms
Aug 06, 2026 6:36:35 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/kms] = path [/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786041246094603-20232-0/minicluster-data/ranger-kms/ews/webapp] .....
Aug 06, 2026 6:36:36 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/kms] = path [/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786041246094603-20232-0/minicluster-data/ranger-kms/ews/webapp].
Aug 06, 2026 6:36:36 PM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : minirangerkms
I20260806 18:36:36.827641 20232 mini_ranger_kms.cc:207] Time spent starting Ranger KMS: real 5.142s user 0.002s sys 0.164s
I20260806 18:36:36.827782 20232 mini_ranger_kms.cc:397] {"name":"kuduclusterkey","cipher":"AES/CTR/NoPadding","length":128,"description":"kuduclusterkey"}
I20260806 18:36:36.827863 20232 mini_ranger_kms.cc:401] 127.19.194.20:49915/kms/v1/keys
Aug 06 18:36:36 dist-test-slave-1xrh krb5kdc[32421](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786041396, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.19.194.20@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Aug 06 18:36:37 dist-test-slave-1xrh krb5kdc[32421](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041396, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.19.194.20@KRBTEST.COM for HTTP/127.19.194.20@KRBTEST.COM
Aug 06 18:36:37 dist-test-slave-1xrh krb5kdc[32421](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041396, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.19.194.20@KRBTEST.COM for HTTP/127.19.194.20@KRBTEST.COM
Aug 06 18:36:39 dist-test-slave-1xrh krb5kdc[32421](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041399, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
WARNING: no policy specified for kudu/127.19.194.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.19.194.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.19.194.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786041246094603-20232-0/krb5kdc/kudu_127.19.194.62.keytab.
Entry for principal kudu/127.19.194.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786041246094603-20232-0/krb5kdc/kudu_127.19.194.62.keytab.
I20260806 18:36:39.882681 20232 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJW69tn/build/debug/bin/kudu
/tmp/dist-test-taskJW69tn/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786041246094603-20232-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786041246094603-20232-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786041246094603-20232-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786041246094603-20232-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.19.194.62:37135
--webserver_interface=127.19.194.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786041246094603-20232-0/krb5kdc/kudu_127.19.194.62.keytab
--principal=kudu/127.19.194.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.19.194.20:32849
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.19.194.62:37135
--ranger_config_path=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786041246094603-20232-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.19.194.20:49915/kms
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786041246094603-20232-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786041246094603-20232-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786041246094603-20232-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 18:36:39.993132 427 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 18:36:39.993391 427 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 18:36:39.993448 427 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 18:36:39.997536 427 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 18:36:39.997593 427 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 18:36:39.997613 427 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 18:36:39.997629 427 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 18:36:39.997644 427 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 18:36:40.003027 427 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.194.20:32849
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786041246094603-20232-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786041246094603-20232-0/minicluster-data/master-0/wal
--ranger_kms_url=127.19.194.20:49915/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.19.194.62:37135
--ranger_config_path=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786041246094603-20232-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786041246094603-20232-0/krb5kdc/kudu_127.19.194.62.keytab
--principal=kudu/127.19.194.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.19.194.62:37135
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786041246094603-20232-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.19.194.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.427
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786041246094603-20232-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 17:59:13 UTC on ccycloud.quasar-wvsnzx.root.comops.site
build id 4571028
I20260806 18:36:40.004240 427 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 18:36:40.005247 427 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 18:36:40.014174 433 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:36:40.014235 436 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 18:36:40.014468 427 server_base.cc:1034] running on GCE node
W20260806 18:36:40.014242 434 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 18:36:40.015219 427 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 18:36:40.016335 427 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 18:36:40.017531 427 hybrid_clock.cc:648] HybridClock initialized: now 1786041400017513 us; error 30 us; skew 500 ppm
Aug 06 18:36:40 dist-test-slave-1xrh krb5kdc[32421](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041400, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.194.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 18:36:40.025692 427 init.cc:377] Logged in from keytab as kudu/127.19.194.62@KRBTEST.COM (short username kudu)
I20260806 18:36:40.026621 427 webserver.cc:460] Webserver started at http://127.19.194.62:43871/ using document root <none> and password file <none>
I20260806 18:36:40.027194 427 fs_manager.cc:359] Metadata directory not provided
I20260806 18:36:40.027254 427 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 18:36:40.027432 427 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 06 18:36:40 dist-test-slave-1xrh krb5kdc[32421](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041400, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.194.62@KRBTEST.COM for HTTP/127.19.194.20@KRBTEST.COM
I20260806 18:36:40.079430 427 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786041246094603-20232-0/minicluster-data/master-0/data/instance:
uuid: "6fa8c37742cf4faeab2b7790b6cc84ab"
format_stamp: "Formatted at 2026-08-06 18:36:40 on dist-test-slave-1xrh"
server_key: "eb919c9585973b87df072e419f5ad1bd"
server_key_iv: "84756b12659b59618a9889db98905a0c"
server_key_version: "kuduclusterkey@0"
I20260806 18:36:40.080329 427 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786041246094603-20232-0/minicluster-data/master-0/wal/instance:
uuid: "6fa8c37742cf4faeab2b7790b6cc84ab"
format_stamp: "Formatted at 2026-08-06 18:36:40 on dist-test-slave-1xrh"
server_key: "eb919c9585973b87df072e419f5ad1bd"
server_key_iv: "84756b12659b59618a9889db98905a0c"
server_key_version: "kuduclusterkey@0"
I20260806 18:36:40.085002 427 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.001s sys 0.004s
I20260806 18:36:40.100668 444 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 18:36:40.102173 427 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260806 18:36:40.102335 427 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786041246094603-20232-0/minicluster-data/master-0/data,/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786041246094603-20232-0/minicluster-data/master-0/wal
uuid: "6fa8c37742cf4faeab2b7790b6cc84ab"
format_stamp: "Formatted at 2026-08-06 18:36:40 on dist-test-slave-1xrh"
server_key: "eb919c9585973b87df072e419f5ad1bd"
server_key_iv: "84756b12659b59618a9889db98905a0c"
server_key_version: "kuduclusterkey@0"
I20260806 18:36:40.102490 427 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786041246094603-20232-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786041246094603-20232-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786041246094603-20232-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 18:36:40.116075 427 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 18:36:40.119086 427 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 18:36:40.119346 427 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 18:36:40.127454 427 rpc_server.cc:307] RPC server started. Bound to: 127.19.194.62:37135
I20260806 18:36:40.127543 500 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.194.62:37135 every 8 connection(s)
I20260806 18:36:40.128461 427 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786041246094603-20232-0/minicluster-data/master-0/data/info.pb
I20260806 18:36:40.130347 20232 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJW69tn/build/debug/bin/kudu as pid 427
I20260806 18:36:40.130481 20232 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786041246094603-20232-0/minicluster-data/master-0/wal/instance
I20260806 18:36:40.130815 20232 external_mini_cluster.cc:1428] Setting key c1bbb6bfafbd11adf52d046bb570fb97
I20260806 18:36:40.132040 501 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 18:36:40.139110 501 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6fa8c37742cf4faeab2b7790b6cc84ab: Bootstrap starting.
I20260806 18:36:40.142064 501 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 6fa8c37742cf4faeab2b7790b6cc84ab: Neither blocks nor log segments found. Creating new log.
I20260806 18:36:40.142977 501 log.cc:824] T 00000000000000000000000000000000 P 6fa8c37742cf4faeab2b7790b6cc84ab: Log is configured to *not* fsync() on all Append() calls
Aug 06 18:36:40 dist-test-slave-1xrh krb5kdc[32421](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041399, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.19.194.62@KRBTEST.COM
I20260806 18:36:40.145303 501 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6fa8c37742cf4faeab2b7790b6cc84ab: No bootstrap required, opened a new log
I20260806 18:36:40.149592 505 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:36:40.132289 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46738 (local address 127.19.194.62:37135)
0806 18:36:40.132720 (+ 431us) server_negotiation.cc:206] Beginning negotiation
0806 18:36:40.132738 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0806 18:36:40.136339 (+ 3601us) server_negotiation.cc:407] Connection header received
0806 18:36:40.137196 (+ 857us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:36:40.137274 (+ 78us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:36:40.137598 (+ 324us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:36:40.137920 (+ 322us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:36:40.138735 (+ 815us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:36:40.139905 (+ 1170us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:36:40.140893 (+ 988us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:36:40.141340 (+ 447us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:36:40.141513 (+ 173us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:36:40.144327 (+ 2814us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:36:40.144334 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:36:40.144347 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 18:36:40.144372 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:36:40.146648 (+ 2276us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:36:40.147165 (+ 517us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:36:40.147171 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:36:40.147178 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:36:40.147288 (+ 110us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:36:40.147576 (+ 288us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:36:40.147580 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:36:40.147582 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:36:40.147971 (+ 389us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:36:40.148209 (+ 238us) server_negotiation.cc:1035] Waiting for connection context
0806 18:36:40.148524 (+ 315us) server_negotiation.cc:299] Negotiation successful
0806 18:36:40.148834 (+ 310us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":283,"thread_start_us":140,"threads_started":1}
I20260806 18:36:40.149402 501 raft_consensus.cc:348] T 00000000000000000000000000000000 P 6fa8c37742cf4faeab2b7790b6cc84ab [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6fa8c37742cf4faeab2b7790b6cc84ab" member_type: VOTER last_known_addr { host: "127.19.194.62" port: 37135 } }
I20260806 18:36:40.149724 501 raft_consensus.cc:374] T 00000000000000000000000000000000 P 6fa8c37742cf4faeab2b7790b6cc84ab [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 18:36:40.149775 501 raft_consensus.cc:729] T 00000000000000000000000000000000 P 6fa8c37742cf4faeab2b7790b6cc84ab [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6fa8c37742cf4faeab2b7790b6cc84ab, State: Initialized, Role: FOLLOWER
I20260806 18:36:40.150297 501 consensus_queue.cc:260] T 00000000000000000000000000000000 P 6fa8c37742cf4faeab2b7790b6cc84ab [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6fa8c37742cf4faeab2b7790b6cc84ab" member_type: VOTER last_known_addr { host: "127.19.194.62" port: 37135 } }
I20260806 18:36:40.150452 501 raft_consensus.cc:388] T 00000000000000000000000000000000 P 6fa8c37742cf4faeab2b7790b6cc84ab [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 18:36:40.150568 501 raft_consensus.cc:482] T 00000000000000000000000000000000 P 6fa8c37742cf4faeab2b7790b6cc84ab [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 18:36:40.150707 501 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 6fa8c37742cf4faeab2b7790b6cc84ab [term 0 FOLLOWER]: Advancing to term 1
I20260806 18:36:40.151918 501 raft_consensus.cc:504] T 00000000000000000000000000000000 P 6fa8c37742cf4faeab2b7790b6cc84ab [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6fa8c37742cf4faeab2b7790b6cc84ab" member_type: VOTER last_known_addr { host: "127.19.194.62" port: 37135 } }
I20260806 18:36:40.152285 501 leader_election.cc:302] T 00000000000000000000000000000000 P 6fa8c37742cf4faeab2b7790b6cc84ab [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 6fa8c37742cf4faeab2b7790b6cc84ab; no voters:
I20260806 18:36:40.152707 501 leader_election.cc:288] T 00000000000000000000000000000000 P 6fa8c37742cf4faeab2b7790b6cc84ab [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 18:36:40.152844 507 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 6fa8c37742cf4faeab2b7790b6cc84ab [term 1 FOLLOWER]: Leader election won for term 1
I20260806 18:36:40.153105 507 raft_consensus.cc:686] T 00000000000000000000000000000000 P 6fa8c37742cf4faeab2b7790b6cc84ab [term 1 LEADER]: Becoming Leader. State: Replica: 6fa8c37742cf4faeab2b7790b6cc84ab, State: Running, Role: LEADER
I20260806 18:36:40.153460 507 consensus_queue.cc:237] T 00000000000000000000000000000000 P 6fa8c37742cf4faeab2b7790b6cc84ab [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6fa8c37742cf4faeab2b7790b6cc84ab" member_type: VOTER last_known_addr { host: "127.19.194.62" port: 37135 } }
I20260806 18:36:40.153982 501 sys_catalog.cc:564] T 00000000000000000000000000000000 P 6fa8c37742cf4faeab2b7790b6cc84ab [sys.catalog]: configured and running, proceeding with master startup.
I20260806 18:36:40.155107 509 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6fa8c37742cf4faeab2b7790b6cc84ab [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "6fa8c37742cf4faeab2b7790b6cc84ab" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6fa8c37742cf4faeab2b7790b6cc84ab" member_type: VOTER last_known_addr { host: "127.19.194.62" port: 37135 } } }
I20260806 18:36:40.155350 509 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6fa8c37742cf4faeab2b7790b6cc84ab [sys.catalog]: This master's current role is: LEADER
I20260806 18:36:40.155646 509 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6fa8c37742cf4faeab2b7790b6cc84ab [sys.catalog]: SysCatalogTable state changed. Reason: New leader 6fa8c37742cf4faeab2b7790b6cc84ab. Latest consensus state: current_term: 1 leader_uuid: "6fa8c37742cf4faeab2b7790b6cc84ab" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6fa8c37742cf4faeab2b7790b6cc84ab" member_type: VOTER last_known_addr { host: "127.19.194.62" port: 37135 } } }
I20260806 18:36:40.155800 509 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6fa8c37742cf4faeab2b7790b6cc84ab [sys.catalog]: This master's current role is: LEADER
I20260806 18:36:40.156253 501 ranger_client.cc:318] Using new properties file: /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786041246094603-20232-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
I20260806 18:36:40.156558 517 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 18:36:40.162889 517 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 18:36:40.174386 517 catalog_manager.cc:1347] Generated new cluster ID: d2fdf2a0e4734b6ca2fb3e3301faa42f
I20260806 18:36:40.174566 517 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 18:36:40.193619 517 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 18:36:40.195552 517 catalog_manager.cc:1504] Loading token signing keys...
I20260806 18:36:40.224251 517 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 6fa8c37742cf4faeab2b7790b6cc84ab: Generated new TSK 0
I20260806 18:36:40.225767 517 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.19.194.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.19.194.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.19.194.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786041246094603-20232-0/krb5kdc/kudu_127.19.194.1.keytab.
Entry for principal kudu/127.19.194.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786041246094603-20232-0/krb5kdc/kudu_127.19.194.1.keytab.
I20260806 18:36:40.928800 20232 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJW69tn/build/debug/bin/kudu
/tmp/dist-test-taskJW69tn/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786041246094603-20232-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786041246094603-20232-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786041246094603-20232-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786041246094603-20232-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.19.194.20:49915/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.19.194.1:0
--local_ip_for_outbound_sockets=127.19.194.1
--webserver_interface=127.19.194.1
--webserver_port=0
--tserver_master_addrs=127.19.194.62:37135
--keytab_file=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786041246094603-20232-0/krb5kdc/kudu_127.19.194.1.keytab
--principal=kudu/127.19.194.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.19.194.20:32849
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786041246094603-20232-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786041246094603-20232-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786041246094603-20232-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 18:36:41.072818 548 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 18:36:41.073110 548 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 18:36:41.073215 548 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 18:36:41.077389 548 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 18:36:41.077474 548 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 18:36:41.077566 548 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.19.194.1
I20260806 18:36:41.084983 548 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.194.20:32849
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786041246094603-20232-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786041246094603-20232-0/minicluster-data/ts-0/wal
--ranger_kms_url=127.19.194.20:49915/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786041246094603-20232-0/krb5kdc/kudu_127.19.194.1.keytab
--principal=kudu/127.19.194.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.19.194.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786041246094603-20232-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.19.194.1
--webserver_port=0
--tserver_master_addrs=127.19.194.62:37135
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.548
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.19.194.1
--log_dir=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786041246094603-20232-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 17:59:13 UTC on ccycloud.quasar-wvsnzx.root.comops.site
build id 4571028
I20260806 18:36:41.086233 548 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 18:36:41.087507 548 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 18:36:41.098569 556 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:36:41.098533 559 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:36:41.098551 555 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 18:36:41.099327 548 server_base.cc:1034] running on GCE node
I20260806 18:36:41.099764 548 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 18:36:41.100472 548 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 18:36:41.101641 548 hybrid_clock.cc:648] HybridClock initialized: now 1786041401101589 us; error 69 us; skew 500 ppm
Aug 06 18:36:41 dist-test-slave-1xrh krb5kdc[32421](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041401, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.194.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 18:36:41.104643 548 init.cc:377] Logged in from keytab as kudu/127.19.194.1@KRBTEST.COM (short username kudu)
I20260806 18:36:41.105451 548 webserver.cc:460] Webserver started at http://127.19.194.1:38179/ using document root <none> and password file <none>
I20260806 18:36:41.106118 548 fs_manager.cc:359] Metadata directory not provided
I20260806 18:36:41.106209 548 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 18:36:41.106489 548 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 06 18:36:41 dist-test-slave-1xrh krb5kdc[32421](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041401, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.194.1@KRBTEST.COM for HTTP/127.19.194.20@KRBTEST.COM
I20260806 18:36:41.121915 548 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786041246094603-20232-0/minicluster-data/ts-0/data/instance:
uuid: "7d2986ffe0e94accb9507f13d58c73f3"
format_stamp: "Formatted at 2026-08-06 18:36:41 on dist-test-slave-1xrh"
server_key: "88c0aa990cc3b5c287a41d8285cfbbcf"
server_key_iv: "5c1b5e38ce01c1798dccfb8cce987472"
server_key_version: "kuduclusterkey@0"
I20260806 18:36:41.124097 548 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786041246094603-20232-0/minicluster-data/ts-0/wal/instance:
uuid: "7d2986ffe0e94accb9507f13d58c73f3"
format_stamp: "Formatted at 2026-08-06 18:36:41 on dist-test-slave-1xrh"
server_key: "88c0aa990cc3b5c287a41d8285cfbbcf"
server_key_iv: "5c1b5e38ce01c1798dccfb8cce987472"
server_key_version: "kuduclusterkey@0"
I20260806 18:36:41.129596 548 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260806 18:36:41.152875 566 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 18:36:41.154574 548 fs_manager.cc:725] Time spent opening block manager: real 0.006s user 0.002s sys 0.000s
I20260806 18:36:41.154703 548 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786041246094603-20232-0/minicluster-data/ts-0/data,/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786041246094603-20232-0/minicluster-data/ts-0/wal
uuid: "7d2986ffe0e94accb9507f13d58c73f3"
format_stamp: "Formatted at 2026-08-06 18:36:41 on dist-test-slave-1xrh"
server_key: "88c0aa990cc3b5c287a41d8285cfbbcf"
server_key_iv: "5c1b5e38ce01c1798dccfb8cce987472"
server_key_version: "kuduclusterkey@0"
I20260806 18:36:41.154824 548 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786041246094603-20232-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786041246094603-20232-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786041246094603-20232-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 18:36:41.171844 548 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 18:36:41.175649 548 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 18:36:41.175925 548 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 18:36:41.176659 548 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 18:36:41.177778 548 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 18:36:41.177903 548 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 18:36:41.178040 548 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 18:36:41.178133 548 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 18:36:41.190831 548 rpc_server.cc:307] RPC server started. Bound to: 127.19.194.1:40435
I20260806 18:36:41.192098 548 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786041246094603-20232-0/minicluster-data/ts-0/data/info.pb
I20260806 18:36:41.197429 20232 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJW69tn/build/debug/bin/kudu as pid 548
I20260806 18:36:41.197566 20232 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786041246094603-20232-0/minicluster-data/ts-0/wal/instance
I20260806 18:36:41.197899 20232 external_mini_cluster.cc:1428] Setting key a2ea80b326e99fe8ad8e37a8afe591e5
I20260806 18:36:41.193470 689 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.194.1:40435 every 8 connection(s)
Aug 06 18:36:41 dist-test-slave-1xrh krb5kdc[32421](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041401, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.194.1@KRBTEST.COM for kudu/127.19.194.62@KRBTEST.COM
WARNING: no policy specified for kudu/127.19.194.2@KRBTEST.COM; defaulting to no policy
I20260806 18:36:41.214452 694 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:36:41.197096 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.194.1:33655 (local address 127.19.194.62:37135)
0806 18:36:41.202161 (+ 5065us) server_negotiation.cc:206] Beginning negotiation
0806 18:36:41.202169 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 18:36:41.202183 (+ 14us) server_negotiation.cc:407] Connection header received
0806 18:36:41.202222 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:36:41.202225 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:36:41.202272 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:36:41.202365 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:36:41.203855 (+ 1490us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:36:41.204739 (+ 884us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:36:41.206640 (+ 1901us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:36:41.206997 (+ 357us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:36:41.207080 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:36:41.210214 (+ 3134us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:36:41.210217 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:36:41.210220 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 18:36:41.210244 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:36:41.212048 (+ 1804us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:36:41.212625 (+ 577us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:36:41.212629 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:36:41.212631 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:36:41.212715 (+ 84us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:36:41.213092 (+ 377us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:36:41.213095 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:36:41.213097 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:36:41.213343 (+ 246us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:36:41.213427 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0806 18:36:41.214133 (+ 706us) server_negotiation.cc:299] Negotiation successful
0806 18:36:41.214259 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":4987,"thread_start_us":109,"threads_started":1}
I20260806 18:36:41.216523 693 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:36:41.195977 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.194.62:37135 (local address 127.19.194.1:33655)
0806 18:36:41.200649 (+ 4672us) negotiation.cc:107] Waiting for socket to connect
0806 18:36:41.200707 (+ 58us) client_negotiation.cc:174] Beginning negotiation
0806 18:36:41.201729 (+ 1022us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:36:41.202581 (+ 852us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:36:41.202590 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:36:41.202945 (+ 355us) client_negotiation.cc:189] Negotiated authn=SASL
0806 18:36:41.203638 (+ 693us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:36:41.203658 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:36:41.204917 (+ 1259us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:36:41.204920 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:36:41.206440 (+ 1520us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:36:41.206451 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:36:41.207349 (+ 898us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:36:41.207353 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:36:41.207660 (+ 307us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:36:41.208327 (+ 667us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 18:36:41.208347 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0806 18:36:41.210039 (+ 1692us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 18:36:41.212177 (+ 2138us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:36:41.212183 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:36:41.212198 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:36:41.212511 (+ 313us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:36:41.212813 (+ 302us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:36:41.212816 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:36:41.212818 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:36:41.212987 (+ 169us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:36:41.213552 (+ 565us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 18:36:41.213559 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 18:36:41.213842 (+ 283us) client_negotiation.cc:769] Sending connection context
0806 18:36:41.215051 (+ 1209us) client_negotiation.cc:240] Negotiation successful
0806 18:36:41.215404 (+ 353us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":4425,"thread_start_us":144,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.19.194.2@KRBTEST.COM" created.
I20260806 18:36:41.220422 690 heartbeater.cc:347] Connected to a master server at 127.19.194.62:37135
I20260806 18:36:41.220666 690 heartbeater.cc:464] Registering TS with master...
I20260806 18:36:41.221228 690 heartbeater.cc:511] Master 127.19.194.62:37135 requested a full tablet report, sending...
I20260806 18:36:41.222951 462 ts_manager.cc:194] Registered new tserver with Master: 7d2986ffe0e94accb9507f13d58c73f3 (127.19.194.1:40435)
I20260806 18:36:41.224576 462 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.19.194.1@KRBTEST.COM'} at 127.19.194.1:33655
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.19.194.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786041246094603-20232-0/krb5kdc/kudu_127.19.194.2.keytab.
Entry for principal kudu/127.19.194.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786041246094603-20232-0/krb5kdc/kudu_127.19.194.2.keytab.
I20260806 18:36:41.246490 20232 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJW69tn/build/debug/bin/kudu
/tmp/dist-test-taskJW69tn/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786041246094603-20232-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786041246094603-20232-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786041246094603-20232-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786041246094603-20232-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.19.194.20:49915/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.19.194.2:0
--local_ip_for_outbound_sockets=127.19.194.2
--webserver_interface=127.19.194.2
--webserver_port=0
--tserver_master_addrs=127.19.194.62:37135
--keytab_file=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786041246094603-20232-0/krb5kdc/kudu_127.19.194.2.keytab
--principal=kudu/127.19.194.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.19.194.20:32849
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786041246094603-20232-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786041246094603-20232-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786041246094603-20232-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 06 18:36:41 dist-test-slave-1xrh krb5kdc[32421](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786041401, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.194.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
W20260806 18:36:41.389330 698 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 18:36:41.389621 698 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 18:36:41.389721 698 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 18:36:41.393868 698 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 18:36:41.393957 698 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 18:36:41.394057 698 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.19.194.2
I20260806 18:36:41.399772 698 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.194.20:32849
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786041246094603-20232-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786041246094603-20232-0/minicluster-data/ts-1/wal
--ranger_kms_url=127.19.194.20:49915/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786041246094603-20232-0/krb5kdc/kudu_127.19.194.2.keytab
--principal=kudu/127.19.194.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.19.194.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786041246094603-20232-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.19.194.2
--webserver_port=0
--tserver_master_addrs=127.19.194.62:37135
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.698
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.19.194.2
--log_dir=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786041246094603-20232-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 17:59:13 UTC on ccycloud.quasar-wvsnzx.root.comops.site
build id 4571028
I20260806 18:36:41.400858 698 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 18:36:41.401889 698 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 18:36:41.411916 704 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:36:41.411890 707 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:36:41.412599 705 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 18:36:41.412964 698 server_base.cc:1034] running on GCE node
I20260806 18:36:41.413378 698 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 18:36:41.414083 698 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 18:36:41.415208 698 hybrid_clock.cc:648] HybridClock initialized: now 1786041401415174 us; error 50 us; skew 500 ppm
Aug 06 18:36:41 dist-test-slave-1xrh krb5kdc[32421](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041401, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.194.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 18:36:41.418360 698 init.cc:377] Logged in from keytab as kudu/127.19.194.2@KRBTEST.COM (short username kudu)
I20260806 18:36:41.419255 698 webserver.cc:460] Webserver started at http://127.19.194.2:44529/ using document root <none> and password file <none>
I20260806 18:36:41.419925 698 fs_manager.cc:359] Metadata directory not provided
I20260806 18:36:41.420070 698 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 18:36:41.420349 698 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 06 18:36:41 dist-test-slave-1xrh krb5kdc[32421](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041401, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.194.2@KRBTEST.COM for HTTP/127.19.194.20@KRBTEST.COM
I20260806 18:36:41.447712 698 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786041246094603-20232-0/minicluster-data/ts-1/data/instance:
uuid: "896964caeacd4a0a8e99996d6dbf6854"
format_stamp: "Formatted at 2026-08-06 18:36:41 on dist-test-slave-1xrh"
server_key: "ab34bbee19311ec972780faec570626b"
server_key_iv: "414bca491c9977ac053eb01022ad9d00"
server_key_version: "kuduclusterkey@0"
I20260806 18:36:41.448510 698 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786041246094603-20232-0/minicluster-data/ts-1/wal/instance:
uuid: "896964caeacd4a0a8e99996d6dbf6854"
format_stamp: "Formatted at 2026-08-06 18:36:41 on dist-test-slave-1xrh"
server_key: "ab34bbee19311ec972780faec570626b"
server_key_iv: "414bca491c9977ac053eb01022ad9d00"
server_key_version: "kuduclusterkey@0"
I20260806 18:36:41.454833 698 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.001s sys 0.004s
I20260806 18:36:41.481154 715 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 18:36:41.482950 698 fs_manager.cc:725] Time spent opening block manager: real 0.006s user 0.002s sys 0.001s
I20260806 18:36:41.483083 698 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786041246094603-20232-0/minicluster-data/ts-1/data,/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786041246094603-20232-0/minicluster-data/ts-1/wal
uuid: "896964caeacd4a0a8e99996d6dbf6854"
format_stamp: "Formatted at 2026-08-06 18:36:41 on dist-test-slave-1xrh"
server_key: "ab34bbee19311ec972780faec570626b"
server_key_iv: "414bca491c9977ac053eb01022ad9d00"
server_key_version: "kuduclusterkey@0"
I20260806 18:36:41.483243 698 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786041246094603-20232-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786041246094603-20232-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786041246094603-20232-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 18:36:41.499572 698 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 18:36:41.503283 698 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 18:36:41.503537 698 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 18:36:41.504303 698 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 18:36:41.505535 698 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 18:36:41.505651 698 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 18:36:41.505757 698 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 18:36:41.505900 698 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 18:36:41.520634 698 rpc_server.cc:307] RPC server started. Bound to: 127.19.194.2:36241
I20260806 18:36:41.521946 698 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786041246094603-20232-0/minicluster-data/ts-1/data/info.pb
I20260806 18:36:41.523095 20232 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJW69tn/build/debug/bin/kudu as pid 698
I20260806 18:36:41.525710 20232 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786041246094603-20232-0/minicluster-data/ts-1/wal/instance
I20260806 18:36:41.526098 20232 external_mini_cluster.cc:1428] Setting key 811e91c4331b34e358522584ef5a4841
I20260806 18:36:41.523598 837 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.194.2:36241 every 8 connection(s)
Aug 06 18:36:41 dist-test-slave-1xrh krb5kdc[32421](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041401, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.194.2@KRBTEST.COM for kudu/127.19.194.62@KRBTEST.COM
WARNING: no policy specified for kudu/127.19.194.3@KRBTEST.COM; defaulting to no policy
I20260806 18:36:41.551803 694 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:36:41.535163 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.194.2:60833 (local address 127.19.194.62:37135)
0806 18:36:41.535301 (+ 138us) server_negotiation.cc:206] Beginning negotiation
0806 18:36:41.535306 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 18:36:41.536435 (+ 1129us) server_negotiation.cc:407] Connection header received
0806 18:36:41.539251 (+ 2816us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:36:41.539255 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:36:41.539314 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:36:41.539376 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:36:41.541184 (+ 1808us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:36:41.542215 (+ 1031us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:36:41.543957 (+ 1742us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:36:41.544294 (+ 337us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:36:41.544384 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:36:41.547702 (+ 3318us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:36:41.547706 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:36:41.547709 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 18:36:41.547736 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:36:41.549509 (+ 1773us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:36:41.550089 (+ 580us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:36:41.550093 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:36:41.550095 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:36:41.550181 (+ 86us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:36:41.550598 (+ 417us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:36:41.550601 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:36:41.550603 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:36:41.550786 (+ 183us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:36:41.550892 (+ 106us) server_negotiation.cc:1035] Waiting for connection context
0806 18:36:41.551496 (+ 604us) server_negotiation.cc:299] Negotiation successful
0806 18:36:41.551621 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260806 18:36:41.554152 840 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:36:41.533352 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.194.62:37135 (local address 127.19.194.2:60833)
0806 18:36:41.536265 (+ 2913us) negotiation.cc:107] Waiting for socket to connect
0806 18:36:41.536339 (+ 74us) client_negotiation.cc:174] Beginning negotiation
0806 18:36:41.538289 (+ 1950us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:36:41.539809 (+ 1520us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:36:41.539818 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:36:41.540213 (+ 395us) client_negotiation.cc:189] Negotiated authn=SASL
0806 18:36:41.540992 (+ 779us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:36:41.541015 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:36:41.542352 (+ 1337us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:36:41.542357 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:36:41.543764 (+ 1407us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:36:41.543776 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:36:41.544606 (+ 830us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:36:41.544611 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:36:41.544811 (+ 200us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:36:41.545615 (+ 804us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 18:36:41.545654 (+ 39us) client_negotiation.cc:656] calling sasl_client_start()
0806 18:36:41.547507 (+ 1853us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 18:36:41.549663 (+ 2156us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:36:41.549686 (+ 23us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:36:41.549703 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:36:41.549972 (+ 269us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:36:41.550273 (+ 301us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:36:41.550277 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:36:41.550279 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:36:41.550484 (+ 205us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:36:41.550894 (+ 410us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 18:36:41.550900 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 18:36:41.551222 (+ 322us) client_negotiation.cc:769] Sending connection context
0806 18:36:41.551434 (+ 212us) client_negotiation.cc:240] Negotiation successful
0806 18:36:41.552052 (+ 618us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2662,"thread_start_us":141,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.19.194.3@KRBTEST.COM" created.
I20260806 18:36:41.555712 838 heartbeater.cc:347] Connected to a master server at 127.19.194.62:37135
I20260806 18:36:41.555974 838 heartbeater.cc:464] Registering TS with master...
I20260806 18:36:41.556567 838 heartbeater.cc:511] Master 127.19.194.62:37135 requested a full tablet report, sending...
I20260806 18:36:41.557828 462 ts_manager.cc:194] Registered new tserver with Master: 896964caeacd4a0a8e99996d6dbf6854 (127.19.194.2:36241)
I20260806 18:36:41.558714 462 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.19.194.2@KRBTEST.COM'} at 127.19.194.2:60833
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.19.194.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786041246094603-20232-0/krb5kdc/kudu_127.19.194.3.keytab.
Entry for principal kudu/127.19.194.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786041246094603-20232-0/krb5kdc/kudu_127.19.194.3.keytab.
I20260806 18:36:41.574429 20232 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJW69tn/build/debug/bin/kudu
/tmp/dist-test-taskJW69tn/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786041246094603-20232-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786041246094603-20232-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786041246094603-20232-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786041246094603-20232-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.19.194.20:49915/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.19.194.3:0
--local_ip_for_outbound_sockets=127.19.194.3
--webserver_interface=127.19.194.3
--webserver_port=0
--tserver_master_addrs=127.19.194.62:37135
--keytab_file=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786041246094603-20232-0/krb5kdc/kudu_127.19.194.3.keytab
--principal=kudu/127.19.194.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.19.194.20:32849
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786041246094603-20232-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786041246094603-20232-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786041246094603-20232-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 18:36:41.729022 843 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 18:36:41.729473 843 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 18:36:41.729612 843 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 18:36:41.734933 843 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 18:36:41.735028 843 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 18:36:41.735167 843 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.19.194.3
I20260806 18:36:41.743216 843 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.194.20:32849
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786041246094603-20232-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786041246094603-20232-0/minicluster-data/ts-2/wal
--ranger_kms_url=127.19.194.20:49915/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786041246094603-20232-0/krb5kdc/kudu_127.19.194.3.keytab
--principal=kudu/127.19.194.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.19.194.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786041246094603-20232-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.19.194.3
--webserver_port=0
--tserver_master_addrs=127.19.194.62:37135
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.843
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.19.194.3
--log_dir=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786041246094603-20232-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 17:59:13 UTC on ccycloud.quasar-wvsnzx.root.comops.site
build id 4571028
I20260806 18:36:41.744649 843 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 18:36:41.746039 843 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260806 18:36:41.759750 843 server_base.cc:1034] running on GCE node
W20260806 18:36:41.759881 853 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:36:41.760154 851 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:36:41.760835 849 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 18:36:41.761368 843 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 18:36:41.762217 843 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 18:36:41.763396 843 hybrid_clock.cc:648] HybridClock initialized: now 1786041401763319 us; error 93 us; skew 500 ppm
Aug 06 18:36:41 dist-test-slave-1xrh krb5kdc[32421](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041401, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.194.62@KRBTEST.COM for HTTP/127.19.194.20@KRBTEST.COM
Aug 06 18:36:41 dist-test-slave-1xrh krb5kdc[32421](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041401, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.194.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 18:36:41.767113 843 init.cc:377] Logged in from keytab as kudu/127.19.194.3@KRBTEST.COM (short username kudu)
I20260806 18:36:41.768311 843 webserver.cc:460] Webserver started at http://127.19.194.3:37169/ using document root <none> and password file <none>
I20260806 18:36:41.769053 843 fs_manager.cc:359] Metadata directory not provided
I20260806 18:36:41.769157 843 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 18:36:41.769454 843 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 06 18:36:41 dist-test-slave-1xrh krb5kdc[32421](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041401, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.194.3@KRBTEST.COM for HTTP/127.19.194.20@KRBTEST.COM
I20260806 18:36:41.803871 843 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786041246094603-20232-0/minicluster-data/ts-2/data/instance:
uuid: "18bef07dfcd84613b8f6c64e991992a8"
format_stamp: "Formatted at 2026-08-06 18:36:41 on dist-test-slave-1xrh"
server_key: "f2561c0e718c8b198198a0a4d13f9eee"
server_key_iv: "b983dddfac721969041586dac3a440ae"
server_key_version: "kuduclusterkey@0"
I20260806 18:36:41.804710 843 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786041246094603-20232-0/minicluster-data/ts-2/wal/instance:
uuid: "18bef07dfcd84613b8f6c64e991992a8"
format_stamp: "Formatted at 2026-08-06 18:36:41 on dist-test-slave-1xrh"
server_key: "f2561c0e718c8b198198a0a4d13f9eee"
server_key_iv: "b983dddfac721969041586dac3a440ae"
server_key_version: "kuduclusterkey@0"
I20260806 18:36:41.821522 843 fs_manager.cc:691] Time spent creating directory manager: real 0.016s user 0.005s sys 0.000s
I20260806 18:36:41.855521 860 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 18:36:41.857364 843 fs_manager.cc:725] Time spent opening block manager: real 0.015s user 0.004s sys 0.000s
I20260806 18:36:41.857507 843 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786041246094603-20232-0/minicluster-data/ts-2/data,/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786041246094603-20232-0/minicluster-data/ts-2/wal
uuid: "18bef07dfcd84613b8f6c64e991992a8"
format_stamp: "Formatted at 2026-08-06 18:36:41 on dist-test-slave-1xrh"
server_key: "f2561c0e718c8b198198a0a4d13f9eee"
server_key_iv: "b983dddfac721969041586dac3a440ae"
server_key_version: "kuduclusterkey@0"
I20260806 18:36:41.857676 843 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786041246094603-20232-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786041246094603-20232-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786041246094603-20232-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 18:36:41.893702 843 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 18:36:41.897301 843 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 18:36:41.897483 843 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 18:36:41.898128 843 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 18:36:41.899089 843 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 18:36:41.899166 843 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 18:36:41.899225 843 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 18:36:41.899255 843 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 18:36:41.909389 843 rpc_server.cc:307] RPC server started. Bound to: 127.19.194.3:37147
I20260806 18:36:41.910485 843 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786041246094603-20232-0/minicluster-data/ts-2/data/info.pb
I20260806 18:36:41.910938 983 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.194.3:37147 every 8 connection(s)
I20260806 18:36:41.913766 20232 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJW69tn/build/debug/bin/kudu as pid 843
I20260806 18:36:41.913887 20232 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786041246094603-20232-0/minicluster-data/ts-2/wal/instance
I20260806 18:36:41.914196 20232 external_mini_cluster.cc:1428] Setting key d87c36245ba6a133abb28a8efb15b4c4
Aug 06 18:36:41 dist-test-slave-1xrh krb5kdc[32421](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041401, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.194.3@KRBTEST.COM for kudu/127.19.194.62@KRBTEST.COM
I20260806 18:36:41.930555 694 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:36:41.914583 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.194.3:53189 (local address 127.19.194.62:37135)
0806 18:36:41.914714 (+ 131us) server_negotiation.cc:206] Beginning negotiation
0806 18:36:41.914719 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 18:36:41.915571 (+ 852us) server_negotiation.cc:407] Connection header received
0806 18:36:41.916760 (+ 1189us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:36:41.916764 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:36:41.916818 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:36:41.916902 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:36:41.918439 (+ 1537us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:36:41.919330 (+ 891us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:36:41.920994 (+ 1664us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:36:41.921358 (+ 364us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:36:41.921435 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:36:41.924575 (+ 3140us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:36:41.924579 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:36:41.924582 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 18:36:41.924608 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:36:41.926394 (+ 1786us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:36:41.928687 (+ 2293us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:36:41.928691 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:36:41.928694 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:36:41.928780 (+ 86us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:36:41.929213 (+ 433us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:36:41.929218 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:36:41.929221 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:36:41.929484 (+ 263us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:36:41.929575 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0806 18:36:41.930244 (+ 669us) server_negotiation.cc:299] Negotiation successful
0806 18:36:41.930389 (+ 145us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260806 18:36:41.932407 987 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:36:41.913501 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.194.62:37135 (local address 127.19.194.3:53189)
0806 18:36:41.915416 (+ 1915us) negotiation.cc:107] Waiting for socket to connect
0806 18:36:41.915473 (+ 57us) client_negotiation.cc:174] Beginning negotiation
0806 18:36:41.916523 (+ 1050us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:36:41.917222 (+ 699us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:36:41.917231 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:36:41.917585 (+ 354us) client_negotiation.cc:189] Negotiated authn=SASL
0806 18:36:41.918238 (+ 653us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:36:41.918261 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:36:41.919500 (+ 1239us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:36:41.919503 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:36:41.920852 (+ 1349us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:36:41.920864 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:36:41.921634 (+ 770us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:36:41.921638 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:36:41.921824 (+ 186us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:36:41.922566 (+ 742us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 18:36:41.922586 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0806 18:36:41.924389 (+ 1803us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 18:36:41.926559 (+ 2170us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:36:41.926564 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:36:41.926579 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:36:41.928565 (+ 1986us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:36:41.928907 (+ 342us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:36:41.928911 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:36:41.928912 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:36:41.929091 (+ 179us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:36:41.929656 (+ 565us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 18:36:41.929662 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 18:36:41.929964 (+ 302us) client_negotiation.cc:769] Sending connection context
0806 18:36:41.930992 (+ 1028us) client_negotiation.cc:240] Negotiation successful
0806 18:36:41.931367 (+ 375us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1678,"thread_start_us":136,"threads_started":1}
I20260806 18:36:41.934010 984 heartbeater.cc:347] Connected to a master server at 127.19.194.62:37135
I20260806 18:36:41.934243 984 heartbeater.cc:464] Registering TS with master...
I20260806 18:36:41.934792 984 heartbeater.cc:511] Master 127.19.194.62:37135 requested a full tablet report, sending...
I20260806 18:36:41.935987 462 ts_manager.cc:194] Registered new tserver with Master: 18bef07dfcd84613b8f6c64e991992a8 (127.19.194.3:37147)
I20260806 18:36:41.936733 462 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.19.194.3@KRBTEST.COM'} at 127.19.194.3:53189
I20260806 18:36:41.940537 20232 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260806 18:36:41.941277 20232 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJW69tn/build/debug/bin/kudu with pid 548
I20260806 18:36:41.960829 20232 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJW69tn/build/debug/bin/kudu with pid 698
I20260806 18:36:41.974390 20232 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJW69tn/build/debug/bin/kudu with pid 843
I20260806 18:36:41.982137 20232 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJW69tn/build/debug/bin/kudu with pid 427
I20260806 18:36:41.989530 20232 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJW69tn/build/debug/bin/kudu
/tmp/dist-test-taskJW69tn/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786041246094603-20232-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786041246094603-20232-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786041246094603-20232-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786041246094603-20232-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.19.194.62:37135
--webserver_interface=127.19.194.62
--webserver_port=43871
--keytab_file=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786041246094603-20232-0/krb5kdc/kudu_127.19.194.62.keytab
--principal=kudu/127.19.194.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.19.194.20:32849
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.19.194.62:37135
--ranger_config_path=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786041246094603-20232-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.19.194.20:49915/kms
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.19.194.20:49915/kms with env {KRB5CCNAME=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786041246094603-20232-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786041246094603-20232-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786041246094603-20232-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 18:36:42.098860 988 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 18:36:42.099177 988 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 18:36:42.099264 988 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 18:36:42.103157 988 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 18:36:42.103260 988 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 18:36:42.103295 988 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 18:36:42.103338 988 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 18:36:42.103369 988 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 18:36:42.108611 988 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.194.20:32849
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786041246094603-20232-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786041246094603-20232-0/minicluster-data/master-0/wal
--ranger_kms_url=invalid.host:1234/kms,127.19.194.20:49915/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.19.194.62:37135
--ranger_config_path=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786041246094603-20232-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786041246094603-20232-0/krb5kdc/kudu_127.19.194.62.keytab
--principal=kudu/127.19.194.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.19.194.62:37135
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786041246094603-20232-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.19.194.62
--webserver_port=43871
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.988
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786041246094603-20232-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 17:59:13 UTC on ccycloud.quasar-wvsnzx.root.comops.site
build id 4571028
I20260806 18:36:42.109671 988 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 18:36:42.110692 988 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 18:36:42.119107 996 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 18:36:42.119154 988 server_base.cc:1034] running on GCE node
W20260806 18:36:42.119098 993 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:36:42.119175 994 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 18:36:42.119943 988 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 18:36:42.121012 988 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 18:36:42.122207 988 hybrid_clock.cc:648] HybridClock initialized: now 1786041402122188 us; error 29 us; skew 500 ppm
Aug 06 18:36:42 dist-test-slave-1xrh krb5kdc[32421](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041402, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.194.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 18:36:42.125077 988 init.cc:377] Logged in from keytab as kudu/127.19.194.62@KRBTEST.COM (short username kudu)
I20260806 18:36:42.125914 988 webserver.cc:460] Webserver started at http://127.19.194.62:43871/ using document root <none> and password file <none>
I20260806 18:36:42.126438 988 fs_manager.cc:359] Metadata directory not provided
I20260806 18:36:42.126533 988 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Aug 06 18:36:42 dist-test-slave-1xrh krb5kdc[32421](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041402, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.194.62@KRBTEST.COM for HTTP/127.19.194.20@KRBTEST.COM
I20260806 18:36:42.181233 988 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.000s sys 0.003s
I20260806 18:36:42.183753 1004 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 18:36:42.185114 988 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260806 18:36:42.185248 988 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786041246094603-20232-0/minicluster-data/master-0/data,/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786041246094603-20232-0/minicluster-data/master-0/wal
uuid: "6fa8c37742cf4faeab2b7790b6cc84ab"
format_stamp: "Formatted at 2026-08-06 18:36:40 on dist-test-slave-1xrh"
server_key: "eb919c9585973b87df072e419f5ad1bd"
server_key_iv: "84756b12659b59618a9889db98905a0c"
server_key_version: "kuduclusterkey@0"
I20260806 18:36:42.185796 988 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786041246094603-20232-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786041246094603-20232-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786041246094603-20232-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 18:36:42.196573 988 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 18:36:42.199018 988 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 18:36:42.199503 988 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 18:36:42.206313 988 rpc_server.cc:307] RPC server started. Bound to: 127.19.194.62:37135
I20260806 18:36:42.206410 1059 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.194.62:37135 every 8 connection(s)
I20260806 18:36:42.207444 988 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786041246094603-20232-0/minicluster-data/master-0/data/info.pb
I20260806 18:36:42.211630 1060 sys_catalog.cc:263] Verifying existing consensus state
I20260806 18:36:42.213109 1060 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6fa8c37742cf4faeab2b7790b6cc84ab: Bootstrap starting.
I20260806 18:36:42.216871 20232 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJW69tn/build/debug/bin/kudu as pid 988
I20260806 18:36:42.217355 20232 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJW69tn/build/debug/bin/kudu
/tmp/dist-test-taskJW69tn/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786041246094603-20232-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786041246094603-20232-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786041246094603-20232-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786041246094603-20232-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.19.194.20:49915/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.19.194.1:40435
--local_ip_for_outbound_sockets=127.19.194.1
--tserver_master_addrs=127.19.194.62:37135
--webserver_port=38179
--webserver_interface=127.19.194.1
--keytab_file=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786041246094603-20232-0/krb5kdc/kudu_127.19.194.1.keytab
--principal=kudu/127.19.194.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.19.194.20:32849
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.19.194.20:49915/kms with env {KRB5CCNAME=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786041246094603-20232-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786041246094603-20232-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786041246094603-20232-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260806 18:36:42.230782 1060 log.cc:824] T 00000000000000000000000000000000 P 6fa8c37742cf4faeab2b7790b6cc84ab: Log is configured to *not* fsync() on all Append() calls
I20260806 18:36:42.237334 1060 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6fa8c37742cf4faeab2b7790b6cc84ab: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260806 18:36:42.237715 1060 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6fa8c37742cf4faeab2b7790b6cc84ab: Bootstrap complete.
I20260806 18:36:42.241770 1060 raft_consensus.cc:348] T 00000000000000000000000000000000 P 6fa8c37742cf4faeab2b7790b6cc84ab [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6fa8c37742cf4faeab2b7790b6cc84ab" member_type: VOTER last_known_addr { host: "127.19.194.62" port: 37135 } }
I20260806 18:36:42.242136 1060 raft_consensus.cc:729] T 00000000000000000000000000000000 P 6fa8c37742cf4faeab2b7790b6cc84ab [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6fa8c37742cf4faeab2b7790b6cc84ab, State: Initialized, Role: FOLLOWER
I20260806 18:36:42.242642 1060 consensus_queue.cc:260] T 00000000000000000000000000000000 P 6fa8c37742cf4faeab2b7790b6cc84ab [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6fa8c37742cf4faeab2b7790b6cc84ab" member_type: VOTER last_known_addr { host: "127.19.194.62" port: 37135 } }
I20260806 18:36:42.242753 1060 raft_consensus.cc:388] T 00000000000000000000000000000000 P 6fa8c37742cf4faeab2b7790b6cc84ab [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 18:36:42.242837 1060 raft_consensus.cc:482] T 00000000000000000000000000000000 P 6fa8c37742cf4faeab2b7790b6cc84ab [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 18:36:42.242959 1060 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 6fa8c37742cf4faeab2b7790b6cc84ab [term 1 FOLLOWER]: Advancing to term 2
I20260806 18:36:42.244851 1060 raft_consensus.cc:504] T 00000000000000000000000000000000 P 6fa8c37742cf4faeab2b7790b6cc84ab [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6fa8c37742cf4faeab2b7790b6cc84ab" member_type: VOTER last_known_addr { host: "127.19.194.62" port: 37135 } }
I20260806 18:36:42.245183 1060 leader_election.cc:302] T 00000000000000000000000000000000 P 6fa8c37742cf4faeab2b7790b6cc84ab [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 6fa8c37742cf4faeab2b7790b6cc84ab; no voters:
I20260806 18:36:42.245476 1060 leader_election.cc:288] T 00000000000000000000000000000000 P 6fa8c37742cf4faeab2b7790b6cc84ab [CANDIDATE]: Term 2 election: Requested vote from peers
I20260806 18:36:42.245631 1065 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 6fa8c37742cf4faeab2b7790b6cc84ab [term 2 FOLLOWER]: Leader election won for term 2
I20260806 18:36:42.245910 1065 raft_consensus.cc:686] T 00000000000000000000000000000000 P 6fa8c37742cf4faeab2b7790b6cc84ab [term 2 LEADER]: Becoming Leader. State: Replica: 6fa8c37742cf4faeab2b7790b6cc84ab, State: Running, Role: LEADER
I20260806 18:36:42.246254 1065 consensus_queue.cc:237] T 00000000000000000000000000000000 P 6fa8c37742cf4faeab2b7790b6cc84ab [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6fa8c37742cf4faeab2b7790b6cc84ab" member_type: VOTER last_known_addr { host: "127.19.194.62" port: 37135 } }
I20260806 18:36:42.246690 1060 sys_catalog.cc:564] T 00000000000000000000000000000000 P 6fa8c37742cf4faeab2b7790b6cc84ab [sys.catalog]: configured and running, proceeding with master startup.
I20260806 18:36:42.247821 1067 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6fa8c37742cf4faeab2b7790b6cc84ab [sys.catalog]: SysCatalogTable state changed. Reason: New leader 6fa8c37742cf4faeab2b7790b6cc84ab. Latest consensus state: current_term: 2 leader_uuid: "6fa8c37742cf4faeab2b7790b6cc84ab" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6fa8c37742cf4faeab2b7790b6cc84ab" member_type: VOTER last_known_addr { host: "127.19.194.62" port: 37135 } } }
I20260806 18:36:42.247943 1067 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6fa8c37742cf4faeab2b7790b6cc84ab [sys.catalog]: This master's current role is: LEADER
I20260806 18:36:42.248263 1066 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6fa8c37742cf4faeab2b7790b6cc84ab [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "6fa8c37742cf4faeab2b7790b6cc84ab" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6fa8c37742cf4faeab2b7790b6cc84ab" member_type: VOTER last_known_addr { host: "127.19.194.62" port: 37135 } } }
I20260806 18:36:42.248309 1076 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 18:36:42.248374 1066 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6fa8c37742cf4faeab2b7790b6cc84ab [sys.catalog]: This master's current role is: LEADER
I20260806 18:36:42.251843 1076 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 18:36:42.252338 1060 ranger_client.cc:318] Using overwritten properties file: /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786041246094603-20232-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
I20260806 18:36:42.252521 1076 catalog_manager.cc:1259] Loaded cluster ID: d2fdf2a0e4734b6ca2fb3e3301faa42f
I20260806 18:36:42.252590 1076 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 18:36:42.255277 1076 catalog_manager.cc:1504] Loading token signing keys...
I20260806 18:36:42.256011 1076 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 6fa8c37742cf4faeab2b7790b6cc84ab: Loaded TSK: 0
I20260806 18:36:42.257360 1076 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260806 18:36:42.364796 1062 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 18:36:42.365141 1062 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 18:36:42.365222 1062 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 18:36:42.371342 1062 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 18:36:42.371430 1062 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 18:36:42.371513 1062 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.19.194.1
I20260806 18:36:42.377597 1062 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.194.20:32849
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786041246094603-20232-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786041246094603-20232-0/minicluster-data/ts-0/wal
--ranger_kms_url=invalid.host:1234/kms,127.19.194.20:49915/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786041246094603-20232-0/krb5kdc/kudu_127.19.194.1.keytab
--principal=kudu/127.19.194.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.19.194.1:40435
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786041246094603-20232-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.19.194.1
--webserver_port=38179
--tserver_master_addrs=127.19.194.62:37135
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1062
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.19.194.1
--log_dir=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786041246094603-20232-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 17:59:13 UTC on ccycloud.quasar-wvsnzx.root.comops.site
build id 4571028
I20260806 18:36:42.378804 1062 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 18:36:42.379920 1062 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 18:36:42.390719 1102 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 18:36:42.390952 1062 server_base.cc:1034] running on GCE node
W20260806 18:36:42.390719 1101 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:36:42.391247 1104 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 18:36:42.391676 1062 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 18:36:42.392318 1062 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 18:36:42.393503 1062 hybrid_clock.cc:648] HybridClock initialized: now 1786041402393471 us; error 46 us; skew 500 ppm
Aug 06 18:36:42 dist-test-slave-1xrh krb5kdc[32421](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041402, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.194.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 18:36:42.396812 1062 init.cc:377] Logged in from keytab as kudu/127.19.194.1@KRBTEST.COM (short username kudu)
I20260806 18:36:42.397868 1062 webserver.cc:460] Webserver started at http://127.19.194.1:38179/ using document root <none> and password file <none>
I20260806 18:36:42.398572 1062 fs_manager.cc:359] Metadata directory not provided
I20260806 18:36:42.398679 1062 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Aug 06 18:36:42 dist-test-slave-1xrh krb5kdc[32421](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041402, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.194.1@KRBTEST.COM for HTTP/127.19.194.20@KRBTEST.COM
I20260806 18:36:42.421666 1062 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.003s sys 0.000s
I20260806 18:36:42.434207 1113 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 18:36:42.435941 1062 fs_manager.cc:725] Time spent opening block manager: real 0.013s user 0.001s sys 0.000s
I20260806 18:36:42.436087 1062 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786041246094603-20232-0/minicluster-data/ts-0/data,/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786041246094603-20232-0/minicluster-data/ts-0/wal
uuid: "7d2986ffe0e94accb9507f13d58c73f3"
format_stamp: "Formatted at 2026-08-06 18:36:41 on dist-test-slave-1xrh"
server_key: "88c0aa990cc3b5c287a41d8285cfbbcf"
server_key_iv: "5c1b5e38ce01c1798dccfb8cce987472"
server_key_version: "kuduclusterkey@0"
I20260806 18:36:42.436872 1062 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786041246094603-20232-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786041246094603-20232-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786041246094603-20232-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 18:36:42.477653 1062 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 18:36:42.482016 1062 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 18:36:42.482232 1062 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 18:36:42.482987 1062 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 18:36:42.484174 1062 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 18:36:42.484239 1062 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 18:36:42.484300 1062 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 18:36:42.484329 1062 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 18:36:42.495496 1062 rpc_server.cc:307] RPC server started. Bound to: 127.19.194.1:40435
I20260806 18:36:42.495664 1233 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.194.1:40435 every 8 connection(s)
I20260806 18:36:42.496543 1062 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786041246094603-20232-0/minicluster-data/ts-0/data/info.pb
I20260806 18:36:42.504213 20232 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJW69tn/build/debug/bin/kudu as pid 1062
I20260806 18:36:42.504722 20232 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJW69tn/build/debug/bin/kudu
/tmp/dist-test-taskJW69tn/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786041246094603-20232-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786041246094603-20232-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786041246094603-20232-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786041246094603-20232-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.19.194.20:49915/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.19.194.2:36241
--local_ip_for_outbound_sockets=127.19.194.2
--tserver_master_addrs=127.19.194.62:37135
--webserver_port=44529
--webserver_interface=127.19.194.2
--keytab_file=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786041246094603-20232-0/krb5kdc/kudu_127.19.194.2.keytab
--principal=kudu/127.19.194.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.19.194.20:32849
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.19.194.20:49915/kms with env {KRB5CCNAME=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786041246094603-20232-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786041246094603-20232-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786041246094603-20232-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 06 18:36:42 dist-test-slave-1xrh krb5kdc[32421](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041402, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.194.1@KRBTEST.COM for kudu/127.19.194.62@KRBTEST.COM
I20260806 18:36:42.516795 1238 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:36:42.498828 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.194.62:37135 (local address 127.19.194.1:53079)
0806 18:36:42.499667 (+ 839us) negotiation.cc:107] Waiting for socket to connect
0806 18:36:42.499705 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0806 18:36:42.500483 (+ 778us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:36:42.501575 (+ 1092us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:36:42.501582 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:36:42.501906 (+ 324us) client_negotiation.cc:189] Negotiated authn=SASL
0806 18:36:42.502350 (+ 444us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:36:42.502363 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:36:42.503968 (+ 1605us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:36:42.503973 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:36:42.505378 (+ 1405us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:36:42.505394 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:36:42.507683 (+ 2289us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:36:42.507688 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:36:42.507895 (+ 207us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:36:42.508556 (+ 661us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 18:36:42.508576 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0806 18:36:42.510189 (+ 1613us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 18:36:42.512841 (+ 2652us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:36:42.512847 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:36:42.512860 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:36:42.513201 (+ 341us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:36:42.513656 (+ 455us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:36:42.513660 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:36:42.513661 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:36:42.513856 (+ 195us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:36:42.514676 (+ 820us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 18:36:42.514684 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 18:36:42.514990 (+ 306us) client_negotiation.cc:769] Sending connection context
0806 18:36:42.515211 (+ 221us) client_negotiation.cc:240] Negotiation successful
0806 18:36:42.515525 (+ 314us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":643,"thread_start_us":117,"threads_started":1}
I20260806 18:36:42.518548 1236 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:36:42.498708 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.194.1:53079 (local address 127.19.194.62:37135)
0806 18:36:42.499384 (+ 676us) server_negotiation.cc:206] Beginning negotiation
0806 18:36:42.499405 (+ 21us) server_negotiation.cc:399] Waiting for connection header
0806 18:36:42.499828 (+ 423us) server_negotiation.cc:407] Connection header received
0806 18:36:42.500909 (+ 1081us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:36:42.500919 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:36:42.501201 (+ 282us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:36:42.501505 (+ 304us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:36:42.502519 (+ 1014us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:36:42.503704 (+ 1185us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:36:42.506550 (+ 2846us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:36:42.507317 (+ 767us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:36:42.507475 (+ 158us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:36:42.510439 (+ 2964us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:36:42.510446 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:36:42.510461 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 18:36:42.510495 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:36:42.512687 (+ 2192us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:36:42.513371 (+ 684us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:36:42.513378 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:36:42.513383 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:36:42.513533 (+ 150us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:36:42.513977 (+ 444us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:36:42.513980 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:36:42.513982 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:36:42.514460 (+ 478us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:36:42.514600 (+ 140us) server_negotiation.cc:1035] Waiting for connection context
0806 18:36:42.517427 (+ 2827us) server_negotiation.cc:299] Negotiation successful
0806 18:36:42.517708 (+ 281us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":510,"thread_start_us":122,"threads_started":1}
I20260806 18:36:42.522019 1234 heartbeater.cc:347] Connected to a master server at 127.19.194.62:37135
I20260806 18:36:42.522246 1234 heartbeater.cc:464] Registering TS with master...
I20260806 18:36:42.522812 1234 heartbeater.cc:511] Master 127.19.194.62:37135 requested a full tablet report, sending...
W20260806 18:36:42.525213 1234 heartbeater.cc:644] Failed to heartbeat to 127.19.194.62:37135 (0 consecutive failures): Service unavailable: Catalog manager is not initialized. State: Starting
I20260806 18:36:42.525333 1234 heartbeater.cc:464] Registering TS with master...
I20260806 18:36:42.525502 1234 heartbeater.cc:511] Master 127.19.194.62:37135 requested a full tablet report, sending...
I20260806 18:36:42.526512 1234 heartbeater.cc:464] Registering TS with master...
I20260806 18:36:42.526677 1234 heartbeater.cc:511] Master 127.19.194.62:37135 requested a full tablet report, sending...
W20260806 18:36:42.527549 1234 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260806 18:36:42.643996 1239 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 18:36:42.644232 1239 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 18:36:42.644273 1239 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 18:36:42.649519 1239 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 18:36:42.649601 1239 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 18:36:42.649686 1239 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.19.194.2
I20260806 18:36:42.655759 1239 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.194.20:32849
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786041246094603-20232-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786041246094603-20232-0/minicluster-data/ts-1/wal
--ranger_kms_url=invalid.host:1234/kms,127.19.194.20:49915/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786041246094603-20232-0/krb5kdc/kudu_127.19.194.2.keytab
--principal=kudu/127.19.194.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.19.194.2:36241
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786041246094603-20232-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.19.194.2
--webserver_port=44529
--tserver_master_addrs=127.19.194.62:37135
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1239
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.19.194.2
--log_dir=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786041246094603-20232-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 17:59:13 UTC on ccycloud.quasar-wvsnzx.root.comops.site
build id 4571028
I20260806 18:36:42.656894 1239 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 18:36:42.657907 1239 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 18:36:42.670624 1245 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:36:42.670627 1244 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:36:42.671069 1247 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 18:36:42.672014 1239 server_base.cc:1034] running on GCE node
I20260806 18:36:42.672523 1239 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 18:36:42.673174 1239 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 18:36:42.676534 1239 hybrid_clock.cc:648] HybridClock initialized: now 1786041402676473 us; error 55 us; skew 500 ppm
Aug 06 18:36:42 dist-test-slave-1xrh krb5kdc[32421](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041402, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.194.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 18:36:42.680217 1239 init.cc:377] Logged in from keytab as kudu/127.19.194.2@KRBTEST.COM (short username kudu)
I20260806 18:36:42.681241 1239 webserver.cc:460] Webserver started at http://127.19.194.2:44529/ using document root <none> and password file <none>
I20260806 18:36:42.681803 1239 fs_manager.cc:359] Metadata directory not provided
I20260806 18:36:42.681929 1239 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Aug 06 18:36:42 dist-test-slave-1xrh krb5kdc[32421](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041402, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.194.2@KRBTEST.COM for HTTP/127.19.194.20@KRBTEST.COM
I20260806 18:36:42.720949 1239 fs_manager.cc:709] Time spent opening directory manager: real 0.009s user 0.003s sys 0.001s
I20260806 18:36:42.724363 1255 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 18:36:42.726019 1239 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.002s sys 0.000s
I20260806 18:36:42.726162 1239 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786041246094603-20232-0/minicluster-data/ts-1/data,/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786041246094603-20232-0/minicluster-data/ts-1/wal
uuid: "896964caeacd4a0a8e99996d6dbf6854"
format_stamp: "Formatted at 2026-08-06 18:36:41 on dist-test-slave-1xrh"
server_key: "ab34bbee19311ec972780faec570626b"
server_key_iv: "414bca491c9977ac053eb01022ad9d00"
server_key_version: "kuduclusterkey@0"
I20260806 18:36:42.726926 1239 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786041246094603-20232-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786041246094603-20232-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786041246094603-20232-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 18:36:42.750057 1239 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 18:36:42.754153 1239 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 18:36:42.754432 1239 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 18:36:42.755221 1239 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 18:36:42.756493 1239 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 18:36:42.756603 1239 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 18:36:42.756708 1239 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 18:36:42.756776 1239 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 18:36:42.771106 1239 rpc_server.cc:307] RPC server started. Bound to: 127.19.194.2:36241
I20260806 18:36:42.772539 1239 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786041246094603-20232-0/minicluster-data/ts-1/data/info.pb
I20260806 18:36:42.773437 20232 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJW69tn/build/debug/bin/kudu as pid 1239
I20260806 18:36:42.773613 1376 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.194.2:36241 every 8 connection(s)
I20260806 18:36:42.774025 20232 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJW69tn/build/debug/bin/kudu
/tmp/dist-test-taskJW69tn/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786041246094603-20232-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786041246094603-20232-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786041246094603-20232-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786041246094603-20232-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.19.194.20:49915/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.19.194.3:37147
--local_ip_for_outbound_sockets=127.19.194.3
--tserver_master_addrs=127.19.194.62:37135
--webserver_port=37169
--webserver_interface=127.19.194.3
--keytab_file=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786041246094603-20232-0/krb5kdc/kudu_127.19.194.3.keytab
--principal=kudu/127.19.194.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.19.194.20:32849
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.19.194.20:49915/kms with env {KRB5CCNAME=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786041246094603-20232-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786041246094603-20232-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786041246094603-20232-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 06 18:36:42 dist-test-slave-1xrh krb5kdc[32421](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041402, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.194.2@KRBTEST.COM for kudu/127.19.194.62@KRBTEST.COM
I20260806 18:36:42.794900 1236 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:36:42.777871 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.194.2:60361 (local address 127.19.194.62:37135)
0806 18:36:42.778011 (+ 140us) server_negotiation.cc:206] Beginning negotiation
0806 18:36:42.778016 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 18:36:42.778036 (+ 20us) server_negotiation.cc:407] Connection header received
0806 18:36:42.779361 (+ 1325us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:36:42.779365 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:36:42.779434 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:36:42.779497 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:36:42.783817 (+ 4320us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:36:42.784638 (+ 821us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:36:42.786407 (+ 1769us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:36:42.786749 (+ 342us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:36:42.786848 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:36:42.790402 (+ 3554us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:36:42.790406 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:36:42.790409 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 18:36:42.790438 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:36:42.792293 (+ 1855us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:36:42.793012 (+ 719us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:36:42.793016 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:36:42.793018 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:36:42.793107 (+ 89us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:36:42.793519 (+ 412us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:36:42.793522 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:36:42.793524 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:36:42.793751 (+ 227us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:36:42.793854 (+ 103us) server_negotiation.cc:1035] Waiting for connection context
0806 18:36:42.794545 (+ 691us) server_negotiation.cc:299] Negotiation successful
0806 18:36:42.794705 (+ 160us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260806 18:36:42.796896 1380 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:36:42.776724 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.194.62:37135 (local address 127.19.194.2:60361)
0806 18:36:42.777848 (+ 1124us) negotiation.cc:107] Waiting for socket to connect
0806 18:36:42.777913 (+ 65us) client_negotiation.cc:174] Beginning negotiation
0806 18:36:42.779036 (+ 1123us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:36:42.782394 (+ 3358us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:36:42.782404 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:36:42.782818 (+ 414us) client_negotiation.cc:189] Negotiated authn=SASL
0806 18:36:42.783587 (+ 769us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:36:42.783609 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:36:42.784827 (+ 1218us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:36:42.784830 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:36:42.786257 (+ 1427us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:36:42.786269 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:36:42.787067 (+ 798us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:36:42.787070 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:36:42.787302 (+ 232us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:36:42.788217 (+ 915us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 18:36:42.788239 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0806 18:36:42.790070 (+ 1831us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 18:36:42.792508 (+ 2438us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:36:42.792514 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:36:42.792533 (+ 19us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:36:42.792887 (+ 354us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:36:42.793215 (+ 328us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:36:42.793218 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:36:42.793220 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:36:42.793400 (+ 180us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:36:42.793927 (+ 527us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 18:36:42.793933 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 18:36:42.794248 (+ 315us) client_negotiation.cc:769] Sending connection context
0806 18:36:42.795415 (+ 1167us) client_negotiation.cc:240] Negotiation successful
0806 18:36:42.795764 (+ 349us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":387,"thread_start_us":152,"threads_started":1}
I20260806 18:36:42.798556 1377 heartbeater.cc:347] Connected to a master server at 127.19.194.62:37135
I20260806 18:36:42.798808 1377 heartbeater.cc:464] Registering TS with master...
I20260806 18:36:42.799408 1377 heartbeater.cc:511] Master 127.19.194.62:37135 requested a full tablet report, sending...
W20260806 18:36:42.801105 1377 heartbeater.cc:644] Failed to heartbeat to 127.19.194.62:37135 (0 consecutive failures): Service unavailable: Catalog manager is not initialized. State: Starting
I20260806 18:36:42.801260 1377 heartbeater.cc:464] Registering TS with master...
I20260806 18:36:42.801455 1377 heartbeater.cc:511] Master 127.19.194.62:37135 requested a full tablet report, sending...
I20260806 18:36:42.802304 1377 heartbeater.cc:464] Registering TS with master...
I20260806 18:36:42.802497 1377 heartbeater.cc:511] Master 127.19.194.62:37135 requested a full tablet report, sending...
W20260806 18:36:42.803293 1377 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260806 18:36:42.945890 1379 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 18:36:42.946319 1379 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 18:36:42.946388 1379 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 18:36:42.952762 1379 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 18:36:42.952886 1379 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 18:36:42.953032 1379 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.19.194.3
I20260806 18:36:42.960556 1379 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.194.20:32849
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786041246094603-20232-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786041246094603-20232-0/minicluster-data/ts-2/wal
--ranger_kms_url=invalid.host:1234/kms,127.19.194.20:49915/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786041246094603-20232-0/krb5kdc/kudu_127.19.194.3.keytab
--principal=kudu/127.19.194.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.19.194.3:37147
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786041246094603-20232-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.19.194.3
--webserver_port=37169
--tserver_master_addrs=127.19.194.62:37135
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1379
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.19.194.3
--log_dir=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786041246094603-20232-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 17:59:13 UTC on ccycloud.quasar-wvsnzx.root.comops.site
build id 4571028
I20260806 18:36:42.962093 1379 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 18:36:42.963371 1379 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 18:36:42.975250 1385 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:36:42.975433 1386 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:36:42.975272 1388 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 18:36:42.975934 1379 server_base.cc:1034] running on GCE node
I20260806 18:36:42.976399 1379 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 18:36:42.977165 1379 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 18:36:42.980468 1379 hybrid_clock.cc:648] HybridClock initialized: now 1786041402980213 us; error 263 us; skew 500 ppm
Aug 06 18:36:42 dist-test-slave-1xrh krb5kdc[32421](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041402, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.194.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 18:36:42.984166 1379 init.cc:377] Logged in from keytab as kudu/127.19.194.3@KRBTEST.COM (short username kudu)
I20260806 18:36:42.985447 1379 webserver.cc:460] Webserver started at http://127.19.194.3:37169/ using document root <none> and password file <none>
I20260806 18:36:42.986198 1379 fs_manager.cc:359] Metadata directory not provided
I20260806 18:36:42.986374 1379 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Aug 06 18:36:42 dist-test-slave-1xrh krb5kdc[32421](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041402, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.194.3@KRBTEST.COM for HTTP/127.19.194.20@KRBTEST.COM
I20260806 18:36:43.012831 1379 fs_manager.cc:709] Time spent opening directory manager: real 0.005s user 0.002s sys 0.001s
I20260806 18:36:43.016995 1396 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 18:36:43.018514 1379 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.002s sys 0.000s
I20260806 18:36:43.018635 1379 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786041246094603-20232-0/minicluster-data/ts-2/data,/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786041246094603-20232-0/minicluster-data/ts-2/wal
uuid: "18bef07dfcd84613b8f6c64e991992a8"
format_stamp: "Formatted at 2026-08-06 18:36:41 on dist-test-slave-1xrh"
server_key: "f2561c0e718c8b198198a0a4d13f9eee"
server_key_iv: "b983dddfac721969041586dac3a440ae"
server_key_version: "kuduclusterkey@0"
I20260806 18:36:43.019366 1379 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786041246094603-20232-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786041246094603-20232-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786041246094603-20232-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 18:36:43.051436 1379 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 18:36:43.055338 1379 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 18:36:43.055619 1379 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 18:36:43.056483 1379 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 18:36:43.057610 1379 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 18:36:43.057734 1379 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 18:36:43.057833 1379 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 18:36:43.057896 1379 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 18:36:43.069625 1379 rpc_server.cc:307] RPC server started. Bound to: 127.19.194.3:37147
I20260806 18:36:43.069706 1517 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.194.3:37147 every 8 connection(s)
I20260806 18:36:43.071578 1379 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786041246094603-20232-0/minicluster-data/ts-2/data/info.pb
I20260806 18:36:43.073194 20232 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJW69tn/build/debug/bin/kudu as pid 1379
I20260806 18:36:43.092382 1236 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:36:43.074014 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46740 (local address 127.19.194.62:37135)
0806 18:36:43.074158 (+ 144us) server_negotiation.cc:206] Beginning negotiation
0806 18:36:43.074162 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 18:36:43.076460 (+ 2298us) server_negotiation.cc:407] Connection header received
0806 18:36:43.076525 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:36:43.076528 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:36:43.076589 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:36:43.076691 (+ 102us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:36:43.077632 (+ 941us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:36:43.078453 (+ 821us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:36:43.084606 (+ 6153us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:36:43.084942 (+ 336us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:36:43.085030 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:36:43.087069 (+ 2039us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:36:43.087073 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:36:43.087076 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 18:36:43.087104 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:36:43.090152 (+ 3048us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:36:43.090691 (+ 539us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:36:43.090754 (+ 63us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:36:43.090756 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:36:43.090861 (+ 105us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:36:43.091246 (+ 385us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:36:43.091250 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:36:43.091251 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:36:43.091499 (+ 248us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:36:43.091672 (+ 173us) server_negotiation.cc:1035] Waiting for connection context
0806 18:36:43.092056 (+ 384us) server_negotiation.cc:299] Negotiation successful
0806 18:36:43.092176 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
Aug 06 18:36:43 dist-test-slave-1xrh krb5kdc[32421](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041402, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.194.3@KRBTEST.COM for kudu/127.19.194.62@KRBTEST.COM
I20260806 18:36:43.107656 1522 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:36:43.076958 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.194.3:45081 (local address 127.19.194.62:37135)
0806 18:36:43.083327 (+ 6369us) server_negotiation.cc:206] Beginning negotiation
0806 18:36:43.083340 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0806 18:36:43.083356 (+ 16us) server_negotiation.cc:407] Connection header received
0806 18:36:43.088159 (+ 4803us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:36:43.088164 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:36:43.088215 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:36:43.088291 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:36:43.096172 (+ 7881us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:36:43.097036 (+ 864us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:36:43.098747 (+ 1711us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:36:43.099159 (+ 412us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:36:43.099271 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:36:43.102040 (+ 2769us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:36:43.102044 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:36:43.102047 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 18:36:43.102073 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:36:43.104023 (+ 1950us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:36:43.104681 (+ 658us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:36:43.104686 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:36:43.104688 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:36:43.104786 (+ 98us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:36:43.105241 (+ 455us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:36:43.105245 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:36:43.105247 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:36:43.105489 (+ 242us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:36:43.105592 (+ 103us) server_negotiation.cc:1035] Waiting for connection context
0806 18:36:43.107319 (+ 1727us) server_negotiation.cc:299] Negotiation successful
0806 18:36:43.107463 (+ 144us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":6283,"thread_start_us":75,"threads_started":1}
I20260806 18:36:43.107787 1521 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:36:43.075541 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.194.62:37135 (local address 127.19.194.3:45081)
0806 18:36:43.077721 (+ 2180us) negotiation.cc:107] Waiting for socket to connect
0806 18:36:43.077777 (+ 56us) client_negotiation.cc:174] Beginning negotiation
0806 18:36:43.087785 (+ 10008us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:36:43.088594 (+ 809us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:36:43.088602 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:36:43.095235 (+ 6633us) client_negotiation.cc:189] Negotiated authn=SASL
0806 18:36:43.095974 (+ 739us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:36:43.096000 (+ 26us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:36:43.097187 (+ 1187us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:36:43.097191 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:36:43.098602 (+ 1411us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:36:43.098614 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:36:43.099333 (+ 719us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:36:43.099337 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:36:43.099531 (+ 194us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:36:43.100136 (+ 605us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 18:36:43.100156 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0806 18:36:43.101850 (+ 1694us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 18:36:43.104163 (+ 2313us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:36:43.104170 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:36:43.104186 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:36:43.104557 (+ 371us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:36:43.104913 (+ 356us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:36:43.104917 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:36:43.104919 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:36:43.105117 (+ 198us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:36:43.105653 (+ 536us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 18:36:43.105660 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 18:36:43.105984 (+ 324us) client_negotiation.cc:769] Sending connection context
0806 18:36:43.106215 (+ 231us) client_negotiation.cc:240] Negotiation successful
0806 18:36:43.106575 (+ 360us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1006,"thread_start_us":131,"threads_started":1}
I20260806 18:36:43.109136 1518 heartbeater.cc:347] Connected to a master server at 127.19.194.62:37135
I20260806 18:36:43.109417 1518 heartbeater.cc:464] Registering TS with master...
I20260806 18:36:43.110021 1518 heartbeater.cc:511] Master 127.19.194.62:37135 requested a full tablet report, sending...
I20260806 18:36:43.111519 1021 ts_manager.cc:194] Registered new tserver with Master: 18bef07dfcd84613b8f6c64e991992a8 (127.19.194.3:37147)
I20260806 18:36:43.113019 1021 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.19.194.3@KRBTEST.COM'} at 127.19.194.3:45081
Aug 06 18:36:43 dist-test-slave-1xrh krb5kdc[32421](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786041403, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.194.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 18:36:43.527763 1234 heartbeater.cc:464] Registering TS with master...
I20260806 18:36:43.528064 1234 heartbeater.cc:511] Master 127.19.194.62:37135 requested a full tablet report, sending...
I20260806 18:36:43.528865 1021 ts_manager.cc:194] Registered new tserver with Master: 7d2986ffe0e94accb9507f13d58c73f3 (127.19.194.1:40435)
I20260806 18:36:43.529699 1021 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.19.194.1@KRBTEST.COM'} at 127.19.194.1:53079
I20260806 18:36:43.803534 1377 heartbeater.cc:464] Registering TS with master...
I20260806 18:36:43.803889 1377 heartbeater.cc:511] Master 127.19.194.62:37135 requested a full tablet report, sending...
I20260806 18:36:43.804694 1021 ts_manager.cc:194] Registered new tserver with Master: 896964caeacd4a0a8e99996d6dbf6854 (127.19.194.2:36241)
I20260806 18:36:43.805529 1021 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.19.194.2@KRBTEST.COM'} at 127.19.194.2:60361
I20260806 18:36:43.809273 20232 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 06 18:36:43 dist-test-slave-1xrh krb5kdc[32421](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041403, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.194.62@KRBTEST.COM for HTTP/127.19.194.20@KRBTEST.COM
I20260806 18:36:43.820933 1542 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:36:43.811177 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46746 (local address 127.19.194.62:37135)
0806 18:36:43.811463 (+ 286us) server_negotiation.cc:206] Beginning negotiation
0806 18:36:43.811468 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 18:36:43.811483 (+ 15us) server_negotiation.cc:407] Connection header received
0806 18:36:43.811523 (+ 40us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:36:43.811526 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:36:43.811574 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:36:43.811673 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:36:43.812564 (+ 891us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:36:43.813430 (+ 866us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:36:43.814599 (+ 1169us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:36:43.814967 (+ 368us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:36:43.815078 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:36:43.816869 (+ 1791us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:36:43.816873 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:36:43.816876 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 18:36:43.816905 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:36:43.818802 (+ 1897us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:36:43.819358 (+ 556us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:36:43.819362 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:36:43.819363 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:36:43.819460 (+ 97us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:36:43.819798 (+ 338us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:36:43.819801 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:36:43.819803 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:36:43.820050 (+ 247us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:36:43.820193 (+ 143us) server_negotiation.cc:1035] Waiting for connection context
0806 18:36:43.820572 (+ 379us) server_negotiation.cc:299] Negotiation successful
0806 18:36:43.820706 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":178,"thread_start_us":97,"threads_started":1}
I20260806 18:36:43.825206 1021 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:46746:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260806 18:36:43.831805 1021 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 18:36:43.868459 1549 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:36:43.851931 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.194.1:40435 (local address 127.0.0.1:46004)
0806 18:36:43.859388 (+ 7457us) negotiation.cc:107] Waiting for socket to connect
0806 18:36:43.859447 (+ 59us) client_negotiation.cc:174] Beginning negotiation
0806 18:36:43.859602 (+ 155us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:36:43.861730 (+ 2128us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:36:43.861736 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:36:43.861763 (+ 27us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 18:36:43.861928 (+ 165us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:36:43.861939 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:36:43.865673 (+ 3734us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:36:43.865679 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:36:43.866750 (+ 1071us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:36:43.866762 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:36:43.867863 (+ 1101us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:36:43.867867 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:36:43.867986 (+ 119us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:36:43.868032 (+ 46us) client_negotiation.cc:769] Sending connection context
0806 18:36:43.868146 (+ 114us) client_negotiation.cc:240] Negotiation successful
0806 18:36:43.868273 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":7275,"spinlock_wait_cycles":4992,"thread_start_us":80,"threads_started":1}
I20260806 18:36:43.868484 1548 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:36:43.852064 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46004 (local address 127.19.194.1:40435)
0806 18:36:43.860200 (+ 8136us) server_negotiation.cc:206] Beginning negotiation
0806 18:36:43.860206 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 18:36:43.860246 (+ 40us) server_negotiation.cc:407] Connection header received
0806 18:36:43.860324 (+ 78us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:36:43.861394 (+ 1070us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:36:43.861579 (+ 185us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:36:43.861721 (+ 142us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 18:36:43.862069 (+ 348us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:36:43.865568 (+ 3499us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:36:43.866901 (+ 1333us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:36:43.867735 (+ 834us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:36:43.867851 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:36:43.868000 (+ 149us) server_negotiation.cc:1035] Waiting for connection context
0806 18:36:43.868220 (+ 220us) server_negotiation.cc:299] Negotiation successful
0806 18:36:43.868335 (+ 115us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":8021,"thread_start_us":145,"threads_started":1}
I20260806 18:36:43.874737 1552 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:36:43.853974 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60600 (local address 127.19.194.2:36241)
0806 18:36:43.854276 (+ 302us) server_negotiation.cc:206] Beginning negotiation
0806 18:36:43.854283 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 18:36:43.869752 (+ 15469us) server_negotiation.cc:407] Connection header received
0806 18:36:43.869837 (+ 85us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:36:43.869843 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:36:43.870037 (+ 194us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:36:43.870168 (+ 131us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 18:36:43.870563 (+ 395us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:36:43.871750 (+ 1187us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:36:43.873140 (+ 1390us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:36:43.874097 (+ 957us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:36:43.874215 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:36:43.874361 (+ 146us) server_negotiation.cc:1035] Waiting for connection context
0806 18:36:43.874489 (+ 128us) server_negotiation.cc:299] Negotiation successful
0806 18:36:43.874590 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":198,"thread_start_us":99,"threads_started":1}
I20260806 18:36:43.874976 1549 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:36:43.853788 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.194.2:36241 (local address 127.0.0.1:60600)
0806 18:36:43.868971 (+ 15183us) negotiation.cc:107] Waiting for socket to connect
0806 18:36:43.868984 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0806 18:36:43.869058 (+ 74us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:36:43.870316 (+ 1258us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:36:43.870319 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:36:43.870336 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 18:36:43.870406 (+ 70us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:36:43.870424 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:36:43.871888 (+ 1464us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:36:43.871892 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:36:43.872988 (+ 1096us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:36:43.873000 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:36:43.874241 (+ 1241us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:36:43.874245 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:36:43.874336 (+ 91us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:36:43.874363 (+ 27us) client_negotiation.cc:769] Sending connection context
0806 18:36:43.874459 (+ 96us) client_negotiation.cc:240] Negotiation successful
0806 18:36:43.874539 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":15127,"thread_start_us":92,"threads_started":1}
I20260806 18:36:43.877507 1162 tablet_service.cc:1467] Processing CreateTablet for tablet d222473487044a2da7e7b0f003e61cb4 (DEFAULT_TABLE table=test-table [id=5dfc659b98c84d449d727198477e8cfc]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 18:36:43.878188 1305 tablet_service.cc:1467] Processing CreateTablet for tablet d222473487044a2da7e7b0f003e61cb4 (DEFAULT_TABLE table=test-table [id=5dfc659b98c84d449d727198477e8cfc]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 18:36:43.878779 1162 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d222473487044a2da7e7b0f003e61cb4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 18:36:43.879196 1305 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d222473487044a2da7e7b0f003e61cb4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 18:36:43.887884 1550 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:36:43.853000 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38456 (local address 127.19.194.3:37147)
0806 18:36:43.853315 (+ 315us) server_negotiation.cc:206] Beginning negotiation
0806 18:36:43.853322 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 18:36:43.859540 (+ 6218us) server_negotiation.cc:407] Connection header received
0806 18:36:43.860905 (+ 1365us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:36:43.860913 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:36:43.861133 (+ 220us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:36:43.861248 (+ 115us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 18:36:43.862862 (+ 1614us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:36:43.863968 (+ 1106us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:36:43.865905 (+ 1937us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:36:43.866744 (+ 839us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:36:43.866828 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:36:43.866984 (+ 156us) server_negotiation.cc:1035] Waiting for connection context
0806 18:36:43.887584 (+ 20600us) server_negotiation.cc:299] Negotiation successful
0806 18:36:43.887735 (+ 151us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":204,"thread_start_us":101,"threads_started":1}
I20260806 18:36:43.888293 1547 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:36:43.852564 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.194.3:37147 (local address 127.0.0.1:38456)
0806 18:36:43.859388 (+ 6824us) negotiation.cc:107] Waiting for socket to connect
0806 18:36:43.859443 (+ 55us) client_negotiation.cc:174] Beginning negotiation
0806 18:36:43.859601 (+ 158us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:36:43.861933 (+ 2332us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:36:43.861935 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:36:43.862616 (+ 681us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 18:36:43.862726 (+ 110us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:36:43.862733 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:36:43.864114 (+ 1381us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:36:43.864117 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:36:43.865759 (+ 1642us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:36:43.865770 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:36:43.887296 (+ 21526us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:36:43.887300 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:36:43.887417 (+ 117us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:36:43.887442 (+ 25us) client_negotiation.cc:769] Sending connection context
0806 18:36:43.887521 (+ 79us) client_negotiation.cc:240] Negotiation successful
0806 18:36:43.887598 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":6689,"spinlock_wait_cycles":13056,"thread_start_us":106,"threads_started":1}
I20260806 18:36:43.888572 1554 tablet_bootstrap.cc:492] T d222473487044a2da7e7b0f003e61cb4 P 896964caeacd4a0a8e99996d6dbf6854: Bootstrap starting.
I20260806 18:36:43.891583 1554 tablet_bootstrap.cc:654] T d222473487044a2da7e7b0f003e61cb4 P 896964caeacd4a0a8e99996d6dbf6854: Neither blocks nor log segments found. Creating new log.
I20260806 18:36:43.891232 1442 tablet_service.cc:1467] Processing CreateTablet for tablet d222473487044a2da7e7b0f003e61cb4 (DEFAULT_TABLE table=test-table [id=5dfc659b98c84d449d727198477e8cfc]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 18:36:43.892266 1442 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d222473487044a2da7e7b0f003e61cb4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 18:36:43.892616 1554 log.cc:824] T d222473487044a2da7e7b0f003e61cb4 P 896964caeacd4a0a8e99996d6dbf6854: Log is configured to *not* fsync() on all Append() calls
I20260806 18:36:43.896432 1553 tablet_bootstrap.cc:492] T d222473487044a2da7e7b0f003e61cb4 P 7d2986ffe0e94accb9507f13d58c73f3: Bootstrap starting.
I20260806 18:36:43.898717 1553 tablet_bootstrap.cc:654] T d222473487044a2da7e7b0f003e61cb4 P 7d2986ffe0e94accb9507f13d58c73f3: Neither blocks nor log segments found. Creating new log.
I20260806 18:36:43.899647 1553 log.cc:824] T d222473487044a2da7e7b0f003e61cb4 P 7d2986ffe0e94accb9507f13d58c73f3: Log is configured to *not* fsync() on all Append() calls
I20260806 18:36:43.900372 1556 tablet_bootstrap.cc:492] T d222473487044a2da7e7b0f003e61cb4 P 18bef07dfcd84613b8f6c64e991992a8: Bootstrap starting.
I20260806 18:36:43.902357 1554 tablet_bootstrap.cc:492] T d222473487044a2da7e7b0f003e61cb4 P 896964caeacd4a0a8e99996d6dbf6854: No bootstrap required, opened a new log
I20260806 18:36:43.902494 1556 tablet_bootstrap.cc:654] T d222473487044a2da7e7b0f003e61cb4 P 18bef07dfcd84613b8f6c64e991992a8: Neither blocks nor log segments found. Creating new log.
I20260806 18:36:43.902616 1554 ts_tablet_manager.cc:1397] T d222473487044a2da7e7b0f003e61cb4 P 896964caeacd4a0a8e99996d6dbf6854: Time spent bootstrapping tablet: real 0.014s user 0.006s sys 0.000s
I20260806 18:36:43.903343 1556 log.cc:824] T d222473487044a2da7e7b0f003e61cb4 P 18bef07dfcd84613b8f6c64e991992a8: Log is configured to *not* fsync() on all Append() calls
I20260806 18:36:43.905316 1556 tablet_bootstrap.cc:492] T d222473487044a2da7e7b0f003e61cb4 P 18bef07dfcd84613b8f6c64e991992a8: No bootstrap required, opened a new log
I20260806 18:36:43.905489 1556 ts_tablet_manager.cc:1397] T d222473487044a2da7e7b0f003e61cb4 P 18bef07dfcd84613b8f6c64e991992a8: Time spent bootstrapping tablet: real 0.005s user 0.000s sys 0.004s
I20260806 18:36:43.907675 1554 raft_consensus.cc:348] T d222473487044a2da7e7b0f003e61cb4 P 896964caeacd4a0a8e99996d6dbf6854 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7d2986ffe0e94accb9507f13d58c73f3" member_type: VOTER last_known_addr { host: "127.19.194.1" port: 40435 } } peers { permanent_uuid: "18bef07dfcd84613b8f6c64e991992a8" member_type: VOTER last_known_addr { host: "127.19.194.3" port: 37147 } } peers { permanent_uuid: "896964caeacd4a0a8e99996d6dbf6854" member_type: VOTER last_known_addr { host: "127.19.194.2" port: 36241 } }
I20260806 18:36:43.907954 1554 raft_consensus.cc:374] T d222473487044a2da7e7b0f003e61cb4 P 896964caeacd4a0a8e99996d6dbf6854 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 18:36:43.908004 1554 raft_consensus.cc:729] T d222473487044a2da7e7b0f003e61cb4 P 896964caeacd4a0a8e99996d6dbf6854 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 896964caeacd4a0a8e99996d6dbf6854, State: Initialized, Role: FOLLOWER
I20260806 18:36:43.908524 1554 consensus_queue.cc:260] T d222473487044a2da7e7b0f003e61cb4 P 896964caeacd4a0a8e99996d6dbf6854 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7d2986ffe0e94accb9507f13d58c73f3" member_type: VOTER last_known_addr { host: "127.19.194.1" port: 40435 } } peers { permanent_uuid: "18bef07dfcd84613b8f6c64e991992a8" member_type: VOTER last_known_addr { host: "127.19.194.3" port: 37147 } } peers { permanent_uuid: "896964caeacd4a0a8e99996d6dbf6854" member_type: VOTER last_known_addr { host: "127.19.194.2" port: 36241 } }
I20260806 18:36:43.909648 1554 ts_tablet_manager.cc:1428] T d222473487044a2da7e7b0f003e61cb4 P 896964caeacd4a0a8e99996d6dbf6854: Time spent starting tablet: real 0.007s user 0.006s sys 0.000s
I20260806 18:36:43.910044 1377 heartbeater.cc:503] Master 127.19.194.62:37135 was elected leader, sending a full tablet report...
I20260806 18:36:43.910408 1556 raft_consensus.cc:348] T d222473487044a2da7e7b0f003e61cb4 P 18bef07dfcd84613b8f6c64e991992a8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7d2986ffe0e94accb9507f13d58c73f3" member_type: VOTER last_known_addr { host: "127.19.194.1" port: 40435 } } peers { permanent_uuid: "18bef07dfcd84613b8f6c64e991992a8" member_type: VOTER last_known_addr { host: "127.19.194.3" port: 37147 } } peers { permanent_uuid: "896964caeacd4a0a8e99996d6dbf6854" member_type: VOTER last_known_addr { host: "127.19.194.2" port: 36241 } }
I20260806 18:36:43.910634 1556 raft_consensus.cc:374] T d222473487044a2da7e7b0f003e61cb4 P 18bef07dfcd84613b8f6c64e991992a8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 18:36:43.910678 1556 raft_consensus.cc:729] T d222473487044a2da7e7b0f003e61cb4 P 18bef07dfcd84613b8f6c64e991992a8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 18bef07dfcd84613b8f6c64e991992a8, State: Initialized, Role: FOLLOWER
I20260806 18:36:43.911139 1556 consensus_queue.cc:260] T d222473487044a2da7e7b0f003e61cb4 P 18bef07dfcd84613b8f6c64e991992a8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7d2986ffe0e94accb9507f13d58c73f3" member_type: VOTER last_known_addr { host: "127.19.194.1" port: 40435 } } peers { permanent_uuid: "18bef07dfcd84613b8f6c64e991992a8" member_type: VOTER last_known_addr { host: "127.19.194.3" port: 37147 } } peers { permanent_uuid: "896964caeacd4a0a8e99996d6dbf6854" member_type: VOTER last_known_addr { host: "127.19.194.2" port: 36241 } }
I20260806 18:36:43.912089 1556 ts_tablet_manager.cc:1428] T d222473487044a2da7e7b0f003e61cb4 P 18bef07dfcd84613b8f6c64e991992a8: Time spent starting tablet: real 0.007s user 0.006s sys 0.000s
I20260806 18:36:43.912439 1518 heartbeater.cc:503] Master 127.19.194.62:37135 was elected leader, sending a full tablet report...
I20260806 18:36:43.915845 1553 tablet_bootstrap.cc:492] T d222473487044a2da7e7b0f003e61cb4 P 7d2986ffe0e94accb9507f13d58c73f3: No bootstrap required, opened a new log
I20260806 18:36:43.916100 1553 ts_tablet_manager.cc:1397] T d222473487044a2da7e7b0f003e61cb4 P 7d2986ffe0e94accb9507f13d58c73f3: Time spent bootstrapping tablet: real 0.020s user 0.005s sys 0.000s
I20260806 18:36:43.919687 1553 raft_consensus.cc:348] T d222473487044a2da7e7b0f003e61cb4 P 7d2986ffe0e94accb9507f13d58c73f3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7d2986ffe0e94accb9507f13d58c73f3" member_type: VOTER last_known_addr { host: "127.19.194.1" port: 40435 } } peers { permanent_uuid: "18bef07dfcd84613b8f6c64e991992a8" member_type: VOTER last_known_addr { host: "127.19.194.3" port: 37147 } } peers { permanent_uuid: "896964caeacd4a0a8e99996d6dbf6854" member_type: VOTER last_known_addr { host: "127.19.194.2" port: 36241 } }
I20260806 18:36:43.919914 1553 raft_consensus.cc:374] T d222473487044a2da7e7b0f003e61cb4 P 7d2986ffe0e94accb9507f13d58c73f3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 18:36:43.919978 1553 raft_consensus.cc:729] T d222473487044a2da7e7b0f003e61cb4 P 7d2986ffe0e94accb9507f13d58c73f3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7d2986ffe0e94accb9507f13d58c73f3, State: Initialized, Role: FOLLOWER
I20260806 18:36:43.920421 1553 consensus_queue.cc:260] T d222473487044a2da7e7b0f003e61cb4 P 7d2986ffe0e94accb9507f13d58c73f3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7d2986ffe0e94accb9507f13d58c73f3" member_type: VOTER last_known_addr { host: "127.19.194.1" port: 40435 } } peers { permanent_uuid: "18bef07dfcd84613b8f6c64e991992a8" member_type: VOTER last_known_addr { host: "127.19.194.3" port: 37147 } } peers { permanent_uuid: "896964caeacd4a0a8e99996d6dbf6854" member_type: VOTER last_known_addr { host: "127.19.194.2" port: 36241 } }
I20260806 18:36:43.921036 1234 heartbeater.cc:503] Master 127.19.194.62:37135 was elected leader, sending a full tablet report...
I20260806 18:36:43.922366 1553 ts_tablet_manager.cc:1428] T d222473487044a2da7e7b0f003e61cb4 P 7d2986ffe0e94accb9507f13d58c73f3: Time spent starting tablet: real 0.006s user 0.001s sys 0.003s
I20260806 18:36:43.969926 1559 raft_consensus.cc:482] T d222473487044a2da7e7b0f003e61cb4 P 896964caeacd4a0a8e99996d6dbf6854 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 18:36:43.970166 1559 raft_consensus.cc:504] T d222473487044a2da7e7b0f003e61cb4 P 896964caeacd4a0a8e99996d6dbf6854 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7d2986ffe0e94accb9507f13d58c73f3" member_type: VOTER last_known_addr { host: "127.19.194.1" port: 40435 } } peers { permanent_uuid: "18bef07dfcd84613b8f6c64e991992a8" member_type: VOTER last_known_addr { host: "127.19.194.3" port: 37147 } } peers { permanent_uuid: "896964caeacd4a0a8e99996d6dbf6854" member_type: VOTER last_known_addr { host: "127.19.194.2" port: 36241 } }
I20260806 18:36:43.971457 1559 leader_election.cc:288] T d222473487044a2da7e7b0f003e61cb4 P 896964caeacd4a0a8e99996d6dbf6854 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7d2986ffe0e94accb9507f13d58c73f3 (127.19.194.1:40435), 18bef07dfcd84613b8f6c64e991992a8 (127.19.194.3:37147)
I20260806 18:36:43.974390 1560 raft_consensus.cc:482] T d222473487044a2da7e7b0f003e61cb4 P 18bef07dfcd84613b8f6c64e991992a8 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 18:36:43.974570 1560 raft_consensus.cc:504] T d222473487044a2da7e7b0f003e61cb4 P 18bef07dfcd84613b8f6c64e991992a8 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7d2986ffe0e94accb9507f13d58c73f3" member_type: VOTER last_known_addr { host: "127.19.194.1" port: 40435 } } peers { permanent_uuid: "18bef07dfcd84613b8f6c64e991992a8" member_type: VOTER last_known_addr { host: "127.19.194.3" port: 37147 } } peers { permanent_uuid: "896964caeacd4a0a8e99996d6dbf6854" member_type: VOTER last_known_addr { host: "127.19.194.2" port: 36241 } }
I20260806 18:36:43.975666 1560 leader_election.cc:288] T d222473487044a2da7e7b0f003e61cb4 P 18bef07dfcd84613b8f6c64e991992a8 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7d2986ffe0e94accb9507f13d58c73f3 (127.19.194.1:40435), 896964caeacd4a0a8e99996d6dbf6854 (127.19.194.2:36241)
I20260806 18:36:43.979039 1550 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:36:43.973071 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.194.2:33787 (local address 127.19.194.3:37147)
0806 18:36:43.973222 (+ 151us) server_negotiation.cc:206] Beginning negotiation
0806 18:36:43.973228 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 18:36:43.973244 (+ 16us) server_negotiation.cc:407] Connection header received
0806 18:36:43.973301 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:36:43.973306 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:36:43.973365 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:36:43.973439 (+ 74us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 18:36:43.973941 (+ 502us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:36:43.974794 (+ 853us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:36:43.976547 (+ 1753us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:36:43.977158 (+ 611us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:36:43.977253 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:36:43.977306 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0806 18:36:43.978802 (+ 1496us) server_negotiation.cc:299] Negotiation successful
0806 18:36:43.978916 (+ 114us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260806 18:36:43.979635 1562 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:36:43.971759 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.194.3:37147 (local address 127.19.194.2:33787)
0806 18:36:43.972724 (+ 965us) negotiation.cc:107] Waiting for socket to connect
0806 18:36:43.972742 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0806 18:36:43.972863 (+ 121us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:36:43.973576 (+ 713us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:36:43.973582 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:36:43.973605 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 18:36:43.973781 (+ 176us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:36:43.973790 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:36:43.974950 (+ 1160us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:36:43.974956 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:36:43.976395 (+ 1439us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:36:43.976406 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:36:43.978498 (+ 2092us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:36:43.978502 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:36:43.978647 (+ 145us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:36:43.978672 (+ 25us) client_negotiation.cc:769] Sending connection context
0806 18:36:43.979410 (+ 738us) client_negotiation.cc:240] Negotiation successful
0806 18:36:43.979492 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":879,"thread_start_us":134,"threads_started":1}
I20260806 18:36:43.980389 1467 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d222473487044a2da7e7b0f003e61cb4" candidate_uuid: "896964caeacd4a0a8e99996d6dbf6854" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "18bef07dfcd84613b8f6c64e991992a8" is_pre_election: true
I20260806 18:36:43.980626 1467 raft_consensus.cc:2454] T d222473487044a2da7e7b0f003e61cb4 P 18bef07dfcd84613b8f6c64e991992a8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 896964caeacd4a0a8e99996d6dbf6854 in term 0.
I20260806 18:36:43.981171 1258 leader_election.cc:302] T d222473487044a2da7e7b0f003e61cb4 P 896964caeacd4a0a8e99996d6dbf6854 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 18bef07dfcd84613b8f6c64e991992a8, 896964caeacd4a0a8e99996d6dbf6854; no voters:
I20260806 18:36:43.981551 1559 raft_consensus.cc:2781] T d222473487044a2da7e7b0f003e61cb4 P 896964caeacd4a0a8e99996d6dbf6854 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 18:36:43.981688 1559 raft_consensus.cc:482] T d222473487044a2da7e7b0f003e61cb4 P 896964caeacd4a0a8e99996d6dbf6854 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 18:36:43.981778 1559 raft_consensus.cc:3037] T d222473487044a2da7e7b0f003e61cb4 P 896964caeacd4a0a8e99996d6dbf6854 [term 0 FOLLOWER]: Advancing to term 1
I20260806 18:36:43.983090 1559 raft_consensus.cc:504] T d222473487044a2da7e7b0f003e61cb4 P 896964caeacd4a0a8e99996d6dbf6854 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7d2986ffe0e94accb9507f13d58c73f3" member_type: VOTER last_known_addr { host: "127.19.194.1" port: 40435 } } peers { permanent_uuid: "18bef07dfcd84613b8f6c64e991992a8" member_type: VOTER last_known_addr { host: "127.19.194.3" port: 37147 } } peers { permanent_uuid: "896964caeacd4a0a8e99996d6dbf6854" member_type: VOTER last_known_addr { host: "127.19.194.2" port: 36241 } }
I20260806 18:36:43.983881 1467 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d222473487044a2da7e7b0f003e61cb4" candidate_uuid: "896964caeacd4a0a8e99996d6dbf6854" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "18bef07dfcd84613b8f6c64e991992a8"
I20260806 18:36:43.983991 1467 raft_consensus.cc:3037] T d222473487044a2da7e7b0f003e61cb4 P 18bef07dfcd84613b8f6c64e991992a8 [term 0 FOLLOWER]: Advancing to term 1
I20260806 18:36:43.984226 1548 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:36:43.972295 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.194.2:52687 (local address 127.19.194.1:40435)
0806 18:36:43.972443 (+ 148us) server_negotiation.cc:206] Beginning negotiation
0806 18:36:43.972450 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 18:36:43.980077 (+ 7627us) server_negotiation.cc:407] Connection header received
0806 18:36:43.980256 (+ 179us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:36:43.980259 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:36:43.980302 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:36:43.980345 (+ 43us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 18:36:43.980702 (+ 357us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:36:43.981457 (+ 755us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:36:43.982901 (+ 1444us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:36:43.983489 (+ 588us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:36:43.983591 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:36:43.983640 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0806 18:36:43.984018 (+ 378us) server_negotiation.cc:299] Negotiation successful
0806 18:36:43.984101 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260806 18:36:43.984603 1562 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:36:43.971882 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.194.1:40435 (local address 127.19.194.2:52687)
0806 18:36:43.979984 (+ 8102us) negotiation.cc:107] Waiting for socket to connect
0806 18:36:43.979997 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0806 18:36:43.980151 (+ 154us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:36:43.980478 (+ 327us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:36:43.980481 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:36:43.980498 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 18:36:43.980562 (+ 64us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:36:43.980567 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:36:43.981603 (+ 1036us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:36:43.981606 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:36:43.982746 (+ 1140us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:36:43.982757 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:36:43.983735 (+ 978us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:36:43.983738 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:36:43.983868 (+ 130us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:36:43.983892 (+ 24us) client_negotiation.cc:769] Sending connection context
0806 18:36:43.984405 (+ 513us) client_negotiation.cc:240] Negotiation successful
0806 18:36:43.984480 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":8046,"thread_start_us":111,"threads_started":1}
I20260806 18:36:43.985188 1467 raft_consensus.cc:2454] T d222473487044a2da7e7b0f003e61cb4 P 18bef07dfcd84613b8f6c64e991992a8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 896964caeacd4a0a8e99996d6dbf6854 in term 1.
I20260806 18:36:43.985178 1173 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d222473487044a2da7e7b0f003e61cb4" candidate_uuid: "896964caeacd4a0a8e99996d6dbf6854" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7d2986ffe0e94accb9507f13d58c73f3" is_pre_election: true
I20260806 18:36:43.985298 1172 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d222473487044a2da7e7b0f003e61cb4" candidate_uuid: "896964caeacd4a0a8e99996d6dbf6854" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7d2986ffe0e94accb9507f13d58c73f3"
I20260806 18:36:43.985457 1173 raft_consensus.cc:2454] T d222473487044a2da7e7b0f003e61cb4 P 7d2986ffe0e94accb9507f13d58c73f3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 896964caeacd4a0a8e99996d6dbf6854 in term 0.
I20260806 18:36:43.985754 1258 leader_election.cc:302] T d222473487044a2da7e7b0f003e61cb4 P 896964caeacd4a0a8e99996d6dbf6854 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 18bef07dfcd84613b8f6c64e991992a8, 896964caeacd4a0a8e99996d6dbf6854; no voters:
I20260806 18:36:43.986045 1559 leader_election.cc:288] T d222473487044a2da7e7b0f003e61cb4 P 896964caeacd4a0a8e99996d6dbf6854 [CANDIDATE]: Term 1 election: Requested vote from peers 7d2986ffe0e94accb9507f13d58c73f3 (127.19.194.1:40435), 18bef07dfcd84613b8f6c64e991992a8 (127.19.194.3:37147)
I20260806 18:36:43.986227 1559 raft_consensus.cc:2781] T d222473487044a2da7e7b0f003e61cb4 P 896964caeacd4a0a8e99996d6dbf6854 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 18:36:43.986343 1559 raft_consensus.cc:686] T d222473487044a2da7e7b0f003e61cb4 P 896964caeacd4a0a8e99996d6dbf6854 [term 1 LEADER]: Becoming Leader. State: Replica: 896964caeacd4a0a8e99996d6dbf6854, State: Running, Role: LEADER
I20260806 18:36:43.986716 1559 consensus_queue.cc:237] T d222473487044a2da7e7b0f003e61cb4 P 896964caeacd4a0a8e99996d6dbf6854 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7d2986ffe0e94accb9507f13d58c73f3" member_type: VOTER last_known_addr { host: "127.19.194.1" port: 40435 } } peers { permanent_uuid: "18bef07dfcd84613b8f6c64e991992a8" member_type: VOTER last_known_addr { host: "127.19.194.3" port: 37147 } } peers { permanent_uuid: "896964caeacd4a0a8e99996d6dbf6854" member_type: VOTER last_known_addr { host: "127.19.194.2" port: 36241 } }
I20260806 18:36:43.991508 1552 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:36:43.976984 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.194.3:37395 (local address 127.19.194.2:36241)
0806 18:36:43.977087 (+ 103us) server_negotiation.cc:206] Beginning negotiation
0806 18:36:43.977091 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 18:36:43.977619 (+ 528us) server_negotiation.cc:407] Connection header received
0806 18:36:43.978515 (+ 896us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:36:43.978518 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:36:43.978574 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:36:43.978614 (+ 40us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 18:36:43.987798 (+ 9184us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:36:43.988518 (+ 720us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:36:43.990154 (+ 1636us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:36:43.990735 (+ 581us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:36:43.990825 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:36:43.990881 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0806 18:36:43.991329 (+ 448us) server_negotiation.cc:299] Negotiation successful
0806 18:36:43.991400 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260806 18:36:43.991288 1020 catalog_manager.cc:5495] T d222473487044a2da7e7b0f003e61cb4 P 896964caeacd4a0a8e99996d6dbf6854 reported cstate change: term changed from 0 to 1, leader changed from <none> to 896964caeacd4a0a8e99996d6dbf6854 (127.19.194.2). New cstate: current_term: 1 leader_uuid: "896964caeacd4a0a8e99996d6dbf6854" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7d2986ffe0e94accb9507f13d58c73f3" member_type: VOTER last_known_addr { host: "127.19.194.1" port: 40435 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "18bef07dfcd84613b8f6c64e991992a8" member_type: VOTER last_known_addr { host: "127.19.194.3" port: 37147 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "896964caeacd4a0a8e99996d6dbf6854" member_type: VOTER last_known_addr { host: "127.19.194.2" port: 36241 } health_report { overall_health: HEALTHY } } }
I20260806 18:36:43.991873 1564 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:36:43.976301 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.194.2:36241 (local address 127.19.194.3:37395)
0806 18:36:43.977549 (+ 1248us) negotiation.cc:107] Waiting for socket to connect
0806 18:36:43.977560 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0806 18:36:43.978402 (+ 842us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:36:43.978735 (+ 333us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:36:43.978737 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:36:43.978750 (+ 13us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 18:36:43.978860 (+ 110us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:36:43.978866 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:36:43.988693 (+ 9827us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:36:43.988696 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:36:43.990013 (+ 1317us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:36:43.990025 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:36:43.991025 (+ 1000us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:36:43.991028 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:36:43.991186 (+ 158us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:36:43.991211 (+ 25us) client_negotiation.cc:769] Sending connection context
0806 18:36:43.991631 (+ 420us) client_negotiation.cc:240] Negotiation successful
0806 18:36:43.991710 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1196,"thread_start_us":99,"threads_started":1}
I20260806 18:36:43.992591 1310 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d222473487044a2da7e7b0f003e61cb4" candidate_uuid: "18bef07dfcd84613b8f6c64e991992a8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "896964caeacd4a0a8e99996d6dbf6854" is_pre_election: true
I20260806 18:36:43.994855 1565 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:36:43.975957 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.194.1:40435 (local address 127.19.194.3:43909)
0806 18:36:43.977232 (+ 1275us) negotiation.cc:107] Waiting for socket to connect
0806 18:36:43.977246 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0806 18:36:43.977334 (+ 88us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:36:43.978144 (+ 810us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:36:43.978147 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:36:43.978162 (+ 15us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 18:36:43.978292 (+ 130us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:36:43.978297 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:36:43.986359 (+ 8062us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:36:43.986363 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:36:43.987625 (+ 1262us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:36:43.987638 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:36:43.994413 (+ 6775us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:36:43.994417 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:36:43.994516 (+ 99us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:36:43.994539 (+ 23us) client_negotiation.cc:769] Sending connection context
0806 18:36:43.994629 (+ 90us) client_negotiation.cc:240] Negotiation successful
0806 18:36:43.994706 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1215,"thread_start_us":93,"threads_started":1}
I20260806 18:36:43.995350 1566 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:36:43.976740 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.194.3:43909 (local address 127.19.194.1:40435)
0806 18:36:43.977811 (+ 1071us) server_negotiation.cc:206] Beginning negotiation
0806 18:36:43.977819 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 18:36:43.977832 (+ 13us) server_negotiation.cc:407] Connection header received
0806 18:36:43.977881 (+ 49us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:36:43.977884 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:36:43.977928 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:36:43.978005 (+ 77us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 18:36:43.979005 (+ 1000us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:36:43.979774 (+ 769us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:36:43.993489 (+ 13715us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:36:43.994168 (+ 679us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:36:43.994293 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:36:43.994341 (+ 48us) server_negotiation.cc:1035] Waiting for connection context
0806 18:36:43.995110 (+ 769us) server_negotiation.cc:299] Negotiation successful
0806 18:36:43.995212 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1001,"thread_start_us":104,"threads_started":1}
I20260806 18:36:43.995873 1173 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d222473487044a2da7e7b0f003e61cb4" candidate_uuid: "18bef07dfcd84613b8f6c64e991992a8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7d2986ffe0e94accb9507f13d58c73f3" is_pre_election: true
I20260806 18:36:43.996028 1173 raft_consensus.cc:2454] T d222473487044a2da7e7b0f003e61cb4 P 7d2986ffe0e94accb9507f13d58c73f3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 18bef07dfcd84613b8f6c64e991992a8 in term 0.
I20260806 18:36:43.996393 1398 leader_election.cc:302] T d222473487044a2da7e7b0f003e61cb4 P 18bef07dfcd84613b8f6c64e991992a8 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 18bef07dfcd84613b8f6c64e991992a8, 7d2986ffe0e94accb9507f13d58c73f3; no voters: 896964caeacd4a0a8e99996d6dbf6854
I20260806 18:36:43.996717 1560 raft_consensus.cc:2741] T d222473487044a2da7e7b0f003e61cb4 P 18bef07dfcd84613b8f6c64e991992a8 [term 1 FOLLOWER]: Leader pre-election decision vote started in defunct term 0: won
W20260806 18:36:43.999462 1235 tablet.cc:2369] T d222473487044a2da7e7b0f003e61cb4 P 7d2986ffe0e94accb9507f13d58c73f3: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 18:36:44.018075 1552 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:36:44.013425 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60606 (local address 127.19.194.2:36241)
0806 18:36:44.013577 (+ 152us) server_negotiation.cc:206] Beginning negotiation
0806 18:36:44.013581 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 18:36:44.013597 (+ 16us) server_negotiation.cc:407] Connection header received
0806 18:36:44.013709 (+ 112us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:36:44.013712 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:36:44.013777 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:36:44.013913 (+ 136us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 18:36:44.014198 (+ 285us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:36:44.014962 (+ 764us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:36:44.016321 (+ 1359us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:36:44.016647 (+ 326us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:36:44.016763 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:36:44.017121 (+ 358us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 18:36:44.017658 (+ 537us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 18:36:44.017793 (+ 135us) server_negotiation.cc:1035] Waiting for connection context
0806 18:36:44.017877 (+ 84us) server_negotiation.cc:299] Negotiation successful
0806 18:36:44.017933 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260806 18:36:44.025038 1467 raft_consensus.cc:1261] T d222473487044a2da7e7b0f003e61cb4 P 18bef07dfcd84613b8f6c64e991992a8 [term 1 FOLLOWER]: Refusing update from remote peer 896964caeacd4a0a8e99996d6dbf6854: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 18:36:44.025024 1173 raft_consensus.cc:3037] T d222473487044a2da7e7b0f003e61cb4 P 7d2986ffe0e94accb9507f13d58c73f3 [term 0 FOLLOWER]: Advancing to term 1
W20260806 18:36:44.025589 1378 tablet.cc:2369] T d222473487044a2da7e7b0f003e61cb4 P 896964caeacd4a0a8e99996d6dbf6854: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 18:36:44.025776 1559 consensus_queue.cc:1035] T d222473487044a2da7e7b0f003e61cb4 P 896964caeacd4a0a8e99996d6dbf6854 [LEADER]: Connected to new peer: Peer: permanent_uuid: "18bef07dfcd84613b8f6c64e991992a8" member_type: VOTER last_known_addr { host: "127.19.194.3" port: 37147 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 18:36:44.026631 1173 raft_consensus.cc:1261] T d222473487044a2da7e7b0f003e61cb4 P 7d2986ffe0e94accb9507f13d58c73f3 [term 1 FOLLOWER]: Refusing update from remote peer 896964caeacd4a0a8e99996d6dbf6854: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 18:36:44.031355 1559 consensus_queue.cc:1035] T d222473487044a2da7e7b0f003e61cb4 P 896964caeacd4a0a8e99996d6dbf6854 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7d2986ffe0e94accb9507f13d58c73f3" member_type: VOTER last_known_addr { host: "127.19.194.1" port: 40435 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 18:36:44.035820 1570 mvcc.cc:205] Tried to move back new op lower bound from 7315625590878765056 to 7315625590732828672. Current Snapshot: MvccSnapshot[applied={T|T < 7315625590878765056}]
I20260806 18:36:44.045586 1573 mvcc.cc:205] Tried to move back new op lower bound from 7315625590878765056 to 7315625590732828672. Current Snapshot: MvccSnapshot[applied={T|T < 7315625590878765056}]
I20260806 18:36:44.045845 1571 mvcc.cc:205] Tried to move back new op lower bound from 7315625590878765056 to 7315625590732828672. Current Snapshot: MvccSnapshot[applied={T|T < 7315625590878765056 or (T in {7315625590878765056})}]
I20260806 18:36:44.046278 1021 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:46746:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 18:36:44.046437 1021 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:46746:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 18:36:44.049120 1021 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 6fa8c37742cf4faeab2b7790b6cc84ab: Sending DeleteTablet for 3 replicas of tablet d222473487044a2da7e7b0f003e61cb4
I20260806 18:36:44.050148 1442 tablet_service.cc:1514] Processing DeleteTablet for tablet d222473487044a2da7e7b0f003e61cb4 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 18:36:44 UTC) from {username='kudu', principal='kudu/127.19.194.62@KRBTEST.COM'} at 127.0.0.1:38456
I20260806 18:36:44.050587 1305 tablet_service.cc:1514] Processing DeleteTablet for tablet d222473487044a2da7e7b0f003e61cb4 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 18:36:44 UTC) from {username='kudu', principal='kudu/127.19.194.62@KRBTEST.COM'} at 127.0.0.1:60600
I20260806 18:36:44.051287 1162 tablet_service.cc:1514] Processing DeleteTablet for tablet d222473487044a2da7e7b0f003e61cb4 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 18:36:44 UTC) from {username='kudu', principal='kudu/127.19.194.62@KRBTEST.COM'} at 127.0.0.1:46004
I20260806 18:36:44.052385 20232 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJW69tn/build/debug/bin/kudu with pid 1062
I20260806 18:36:44.062227 1580 tablet_replica.cc:330] T d222473487044a2da7e7b0f003e61cb4 P 896964caeacd4a0a8e99996d6dbf6854: stopping tablet replica
I20260806 18:36:44.062485 1579 tablet_replica.cc:330] T d222473487044a2da7e7b0f003e61cb4 P 18bef07dfcd84613b8f6c64e991992a8: stopping tablet replica
I20260806 18:36:44.062870 1579 raft_consensus.cc:2229] T d222473487044a2da7e7b0f003e61cb4 P 18bef07dfcd84613b8f6c64e991992a8 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 18:36:44.063247 1580 raft_consensus.cc:2229] T d222473487044a2da7e7b0f003e61cb4 P 896964caeacd4a0a8e99996d6dbf6854 [term 1 LEADER]: Raft consensus shutting down.
I20260806 18:36:44.063642 1579 raft_consensus.cc:2258] T d222473487044a2da7e7b0f003e61cb4 P 18bef07dfcd84613b8f6c64e991992a8 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 18:36:44.064143 1580 raft_consensus.cc:2258] T d222473487044a2da7e7b0f003e61cb4 P 896964caeacd4a0a8e99996d6dbf6854 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 18:36:44.066170 1579 ts_tablet_manager.cc:1905] T d222473487044a2da7e7b0f003e61cb4 P 18bef07dfcd84613b8f6c64e991992a8: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 18:36:44.067912 1580 ts_tablet_manager.cc:1905] T d222473487044a2da7e7b0f003e61cb4 P 896964caeacd4a0a8e99996d6dbf6854: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 18:36:44.070613 1580 ts_tablet_manager.cc:1918] T d222473487044a2da7e7b0f003e61cb4 P 896964caeacd4a0a8e99996d6dbf6854: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 18:36:44.073024 1579 ts_tablet_manager.cc:1918] T d222473487044a2da7e7b0f003e61cb4 P 18bef07dfcd84613b8f6c64e991992a8: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 18:36:44.073179 1580 log.cc:1196] T d222473487044a2da7e7b0f003e61cb4 P 896964caeacd4a0a8e99996d6dbf6854: Deleting WAL directory at /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786041246094603-20232-0/minicluster-data/ts-1/wal/wals/d222473487044a2da7e7b0f003e61cb4
I20260806 18:36:44.073570 1580 ts_tablet_manager.cc:1939] T d222473487044a2da7e7b0f003e61cb4 P 896964caeacd4a0a8e99996d6dbf6854: Deleting consensus metadata
I20260806 18:36:44.073709 1579 log.cc:1196] T d222473487044a2da7e7b0f003e61cb4 P 18bef07dfcd84613b8f6c64e991992a8: Deleting WAL directory at /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786041246094603-20232-0/minicluster-data/ts-2/wal/wals/d222473487044a2da7e7b0f003e61cb4
I20260806 18:36:44.074069 1579 ts_tablet_manager.cc:1939] T d222473487044a2da7e7b0f003e61cb4 P 18bef07dfcd84613b8f6c64e991992a8: Deleting consensus metadata
I20260806 18:36:44.075405 1005 catalog_manager.cc:4841] TS 896964caeacd4a0a8e99996d6dbf6854 (127.19.194.2:36241): tablet d222473487044a2da7e7b0f003e61cb4 (table test-table [id=5dfc659b98c84d449d727198477e8cfc]) successfully deleted
I20260806 18:36:44.076752 1007 catalog_manager.cc:4841] TS 18bef07dfcd84613b8f6c64e991992a8 (127.19.194.3:37147): tablet d222473487044a2da7e7b0f003e61cb4 (table test-table [id=5dfc659b98c84d449d727198477e8cfc]) successfully deleted
W20260806 18:36:44.077585 1006 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.19.194.1:40435 (error 108)
I20260806 18:36:44.079079 1551 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:36:44.078742 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.194.1:40435 (local address 127.0.0.1:46018)
0806 18:36:44.078924 (+ 182us) negotiation.cc:107] Waiting for socket to connect
0806 18:36:44.078998 (+ 74us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.194.1:40435: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":92}
W20260806 18:36:44.079362 1006 catalog_manager.cc:4584] TS 7d2986ffe0e94accb9507f13d58c73f3 (127.19.194.1:40435): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet d222473487044a2da7e7b0f003e61cb4: Network error: Client connection negotiation failed: client connection to 127.19.194.1:40435: connect: Connection refused (error 111)
I20260806 18:36:44.079739 20232 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJW69tn/build/debug/bin/kudu with pid 1239
I20260806 18:36:44.090301 20232 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJW69tn/build/debug/bin/kudu with pid 1379
I20260806 18:36:44.098949 20232 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJW69tn/build/debug/bin/kudu with pid 988
I20260806 18:36:44.106492 20232 mini_ranger_kms.cc:60] Stopping Ranger KMS...
I20260806 18:36:44.450937 20232 mini_ranger_kms.cc:62] Stopped Ranger KMS
I20260806 18:36:44.451048 20232 mini_ranger.cc:66] Stopping Ranger...
I20260806 18:36:44.795437 20232 mini_ranger.cc:68] Stopped Ranger
2026-08-06 18:36:44.795 UTC [32454] LOG: received smart shutdown request
2026-08-06 18:36:44.800 UTC [32454] LOG: background worker "logical replication launcher" (PID 32464) exited with exit code 1
2026-08-06 18:36:44.800 UTC [32458] LOG: shutting down
2026-08-06 18:36:44.966 UTC [32454] LOG: database system is shut down
2026-08-06T18:36:44Z chronyd exiting
[ OK ] SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers (48661 ms)
[ RUN ] SecurityITest.IPKICACert
Loading random data
Initializing database '/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786041246094603-20232-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 18:36:45 dist-test-slave-1xrh krb5kdc[1592](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 18:36:45 dist-test-slave-1xrh krb5kdc[1592](info): set up 2 sockets
Aug 06 18:36:45 dist-test-slave-1xrh krb5kdc[1592](info): commencing operation
krb5kdc: starting...
W20260806 18:36:46.156132 20232 mini_kdc.cc:121] Time spent starting KDC: real 1.047s user 0.003s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786041246094603-20232-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786041246094603-20232-0/krb5kdc/test-user.keytab.
Aug 06 18:36:46 dist-test-slave-1xrh krb5kdc[1592](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041406, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T18:36:46Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T18:36:46Z Disabled control of system clock
WARNING: no policy specified for kudu/127.19.194.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.19.194.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.19.194.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786041246094603-20232-0/krb5kdc/kudu_127.19.194.62.keytab.
Entry for principal kudu/127.19.194.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786041246094603-20232-0/krb5kdc/kudu_127.19.194.62.keytab.
I20260806 18:36:46.274345 20232 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJW69tn/build/debug/bin/kudu
/tmp/dist-test-taskJW69tn/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786041246094603-20232-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786041246094603-20232-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786041246094603-20232-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786041246094603-20232-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.19.194.62:35711
--webserver_interface=127.19.194.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786041246094603-20232-0/krb5kdc/kudu_127.19.194.62.keytab
--principal=kudu/127.19.194.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.19.194.20:38415
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.19.194.62:35711,127.19.194.61:37625,127.19.194.60:46189
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786041246094603-20232-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786041246094603-20232-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786041246094603-20232-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 18:36:46.382146 1606 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 18:36:46.382429 1606 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 18:36:46.382496 1606 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 18:36:46.386449 1606 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 18:36:46.386538 1606 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 18:36:46.386592 1606 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 18:36:46.386629 1606 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 18:36:46.386663 1606 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 18:36:46.391479 1606 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.194.20:38415
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786041246094603-20232-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786041246094603-20232-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.19.194.62:35711,127.19.194.61:37625,127.19.194.60:46189
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786041246094603-20232-0/krb5kdc/kudu_127.19.194.62.keytab
--principal=kudu/127.19.194.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.19.194.62:35711
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786041246094603-20232-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.19.194.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1606
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786041246094603-20232-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 17:59:13 UTC on ccycloud.quasar-wvsnzx.root.comops.site
build id 4571028
I20260806 18:36:46.392410 1606 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 18:36:46.393404 1606 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 18:36:46.401495 1612 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:36:46.401638 1611 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 18:36:46.401674 1606 server_base.cc:1034] running on GCE node
W20260806 18:36:46.401507 1614 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 18:36:46.402441 1606 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 18:36:46.403533 1606 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 18:36:46.404699 1606 hybrid_clock.cc:648] HybridClock initialized: now 1786041406404669 us; error 41 us; skew 500 ppm
Aug 06 18:36:46 dist-test-slave-1xrh krb5kdc[1592](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041406, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.194.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 18:36:46.407487 1606 init.cc:377] Logged in from keytab as kudu/127.19.194.62@KRBTEST.COM (short username kudu)
I20260806 18:36:46.408367 1606 webserver.cc:460] Webserver started at http://127.19.194.62:33207/ using document root <none> and password file <none>
I20260806 18:36:46.408897 1606 fs_manager.cc:359] Metadata directory not provided
I20260806 18:36:46.408959 1606 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 18:36:46.409181 1606 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 18:36:46.411089 1606 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786041246094603-20232-0/minicluster-data/master-0/data/instance:
uuid: "168bc8903141442bab766da1d878ea87"
format_stamp: "Formatted at 2026-08-06 18:36:46 on dist-test-slave-1xrh"
server_key: "b753c9938410ea4b793d14aa7d8000dc"
server_key_iv: "0ea69fc1cfca860097f0df6e3e517ef9"
server_key_version: "encryptionkey@0"
I20260806 18:36:46.411701 1606 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786041246094603-20232-0/minicluster-data/master-0/wal/instance:
uuid: "168bc8903141442bab766da1d878ea87"
format_stamp: "Formatted at 2026-08-06 18:36:46 on dist-test-slave-1xrh"
server_key: "b753c9938410ea4b793d14aa7d8000dc"
server_key_iv: "0ea69fc1cfca860097f0df6e3e517ef9"
server_key_version: "encryptionkey@0"
I20260806 18:36:46.415323 1606 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.000s
I20260806 18:36:46.417631 1623 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 18:36:46.418810 1606 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260806 18:36:46.418949 1606 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786041246094603-20232-0/minicluster-data/master-0/data,/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786041246094603-20232-0/minicluster-data/master-0/wal
uuid: "168bc8903141442bab766da1d878ea87"
format_stamp: "Formatted at 2026-08-06 18:36:46 on dist-test-slave-1xrh"
server_key: "b753c9938410ea4b793d14aa7d8000dc"
server_key_iv: "0ea69fc1cfca860097f0df6e3e517ef9"
server_key_version: "encryptionkey@0"
I20260806 18:36:46.419066 1606 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786041246094603-20232-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786041246094603-20232-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786041246094603-20232-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 18:36:46.433539 1606 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 18:36:46.436282 1606 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 18:36:46.436472 1606 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 18:36:46.443485 1606 rpc_server.cc:307] RPC server started. Bound to: 127.19.194.62:35711
I20260806 18:36:46.443513 1677 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.194.62:35711 every 8 connection(s)
I20260806 18:36:46.444391 1606 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786041246094603-20232-0/minicluster-data/master-0/data/info.pb
I20260806 18:36:46.446882 1678 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 18:36:46.449779 20232 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJW69tn/build/debug/bin/kudu as pid 1606
I20260806 18:36:46.449927 20232 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786041246094603-20232-0/minicluster-data/master-0/wal/instance
I20260806 18:36:46.450232 20232 external_mini_cluster.cc:1428] Setting key 9d79e3b9ae3ac06153173e8057aa2af6
I20260806 18:36:46.450822 1678 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.19.194.62" port: 35711 } has no permanent_uuid. Determining permanent_uuid...
WARNING: no policy specified for kudu/127.19.194.61@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.19.194.61@KRBTEST.COM" created.
Aug 06 18:36:46 dist-test-slave-1xrh krb5kdc[1592](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041406, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.194.62@KRBTEST.COM for kudu/127.19.194.62@KRBTEST.COM
I20260806 18:36:46.469112 1681 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:36:46.453279 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.194.62:35711 (local address 127.0.0.1:46276)
0806 18:36:46.453756 (+ 477us) negotiation.cc:107] Waiting for socket to connect
0806 18:36:46.453809 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0806 18:36:46.455041 (+ 1232us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:36:46.455811 (+ 770us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:36:46.455820 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:36:46.456315 (+ 495us) client_negotiation.cc:189] Negotiated authn=SASL
0806 18:36:46.456729 (+ 414us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:36:46.456753 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:36:46.457908 (+ 1155us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:36:46.457913 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:36:46.459183 (+ 1270us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:36:46.459197 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:36:46.459919 (+ 722us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:36:46.459922 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:36:46.460003 (+ 81us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:36:46.460538 (+ 535us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 18:36:46.460579 (+ 41us) client_negotiation.cc:656] calling sasl_client_start()
0806 18:36:46.463873 (+ 3294us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 18:36:46.466257 (+ 2384us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:36:46.466266 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:36:46.466275 (+ 9us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:36:46.466511 (+ 236us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:36:46.466910 (+ 399us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:36:46.466912 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:36:46.466913 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:36:46.467018 (+ 105us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:36:46.467864 (+ 846us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 18:36:46.467869 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 18:36:46.467995 (+ 126us) client_negotiation.cc:769] Sending connection context
0806 18:36:46.468124 (+ 129us) client_negotiation.cc:240] Negotiation successful
0806 18:36:46.468363 (+ 239us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":262,"thread_start_us":98,"threads_started":1}
I20260806 18:36:46.469116 1682 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:36:46.453913 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46276 (local address 127.19.194.62:35711)
0806 18:36:46.454183 (+ 270us) server_negotiation.cc:206] Beginning negotiation
0806 18:36:46.454191 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 18:36:46.454225 (+ 34us) server_negotiation.cc:407] Connection header received
0806 18:36:46.455413 (+ 1188us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:36:46.455418 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:36:46.455589 (+ 171us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:36:46.455730 (+ 141us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:36:46.456914 (+ 1184us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:36:46.457759 (+ 845us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:36:46.459322 (+ 1563us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:36:46.459769 (+ 447us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:36:46.459895 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:36:46.464036 (+ 4141us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:36:46.464042 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:36:46.464052 (+ 10us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 18:36:46.464074 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:36:46.466131 (+ 2057us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:36:46.466645 (+ 514us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:36:46.466654 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:36:46.466659 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:36:46.466804 (+ 145us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:36:46.467162 (+ 358us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:36:46.467169 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:36:46.467173 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:36:46.467731 (+ 558us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:36:46.467873 (+ 142us) server_negotiation.cc:1035] Waiting for connection context
0806 18:36:46.468279 (+ 406us) server_negotiation.cc:299] Negotiation successful
0806 18:36:46.468455 (+ 176us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":172,"thread_start_us":94,"threads_started":1}
I20260806 18:36:46.471747 1678 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.19.194.61" port: 37625 } has no permanent_uuid. Determining permanent_uuid...
I20260806 18:36:46.472465 1681 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:36:46.472194 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.194.61:37625 (local address 127.0.0.1:38548)
0806 18:36:46.472326 (+ 132us) negotiation.cc:107] Waiting for socket to connect
0806 18:36:46.472396 (+ 70us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.194.61:37625: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":51}
W20260806 18:36:46.472675 1628 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.19.194.61:37625: connect: Connection refused (error 111)
I20260806 18:36:46.473579 1681 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:36:46.473302 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.194.61:37625 (local address 127.0.0.1:38562)
0806 18:36:46.473440 (+ 138us) negotiation.cc:107] Waiting for socket to connect
0806 18:36:46.473498 (+ 58us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.194.61:37625: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":43}
W20260806 18:36:46.473737 1678 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.19.194.61:37625: Network error: Client connection negotiation failed: client connection to 127.19.194.61:37625: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.19.194.61 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786041246094603-20232-0/krb5kdc/kudu_127.19.194.61.keytab.
Entry for principal kudu/127.19.194.61 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786041246094603-20232-0/krb5kdc/kudu_127.19.194.61.keytab.
I20260806 18:36:46.480228 20232 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJW69tn/build/debug/bin/kudu
/tmp/dist-test-taskJW69tn/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786041246094603-20232-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786041246094603-20232-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786041246094603-20232-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786041246094603-20232-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.19.194.61:37625
--webserver_interface=127.19.194.61
--webserver_port=0
--keytab_file=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786041246094603-20232-0/krb5kdc/kudu_127.19.194.61.keytab
--principal=kudu/127.19.194.61
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.19.194.20:38415
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.19.194.62:35711,127.19.194.61:37625,127.19.194.60:46189
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786041246094603-20232-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786041246094603-20232-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786041246094603-20232-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260806 18:36:46.522974 1678 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.19.194.61" port: 37625 } attempt: 1
I20260806 18:36:46.523932 1681 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:36:46.523566 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.194.61:37625 (local address 127.0.0.1:38564)
0806 18:36:46.523751 (+ 185us) negotiation.cc:107] Waiting for socket to connect
0806 18:36:46.523837 (+ 86us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.194.61:37625: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":78}
I20260806 18:36:46.524674 1681 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:36:46.524455 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.194.61:37625 (local address 127.0.0.1:38578)
0806 18:36:46.524561 (+ 106us) negotiation.cc:107] Waiting for socket to connect
0806 18:36:46.524612 (+ 51us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.194.61:37625: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":42}
W20260806 18:36:46.524813 1678 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.19.194.61:37625: Network error: Client connection negotiation failed: client connection to 127.19.194.61:37625: connect: Connection refused (error 111)
W20260806 18:36:46.589155 1685 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 18:36:46.589381 1685 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 18:36:46.589419 1685 flags.cc:425] Enabled unsafe flag: --never_fsync=true
I20260806 18:36:46.592975 1678 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.19.194.61" port: 37625 } attempt: 2
W20260806 18:36:46.593326 1685 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 18:36:46.593369 1685 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 18:36:46.593386 1685 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 18:36:46.593429 1685 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 18:36:46.593462 1685 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 18:36:46.593876 1681 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:36:46.593563 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.194.61:37625 (local address 127.0.0.1:38584)
0806 18:36:46.593724 (+ 161us) negotiation.cc:107] Waiting for socket to connect
0806 18:36:46.593791 (+ 67us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.194.61:37625: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":55}
I20260806 18:36:46.594619 1681 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:36:46.594389 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.194.61:37625 (local address 127.0.0.1:38586)
0806 18:36:46.594497 (+ 108us) negotiation.cc:107] Waiting for socket to connect
0806 18:36:46.594552 (+ 55us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.194.61:37625: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":44}
W20260806 18:36:46.594748 1678 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.19.194.61:37625: Network error: Client connection negotiation failed: client connection to 127.19.194.61:37625: connect: Connection refused (error 111)
I20260806 18:36:46.599017 1685 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.194.20:38415
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786041246094603-20232-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786041246094603-20232-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.19.194.62:35711,127.19.194.61:37625,127.19.194.60:46189
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786041246094603-20232-0/krb5kdc/kudu_127.19.194.61.keytab
--principal=kudu/127.19.194.61
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.19.194.61:37625
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786041246094603-20232-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.19.194.61
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1685
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786041246094603-20232-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 17:59:13 UTC on ccycloud.quasar-wvsnzx.root.comops.site
build id 4571028
I20260806 18:36:46.599993 1685 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 18:36:46.600908 1685 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 18:36:46.608742 1690 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:36:46.608814 1691 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:36:46.608742 1693 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 18:36:46.609118 1685 server_base.cc:1034] running on GCE node
I20260806 18:36:46.609638 1685 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 18:36:46.610607 1685 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 18:36:46.611788 1685 hybrid_clock.cc:648] HybridClock initialized: now 1786041406611760 us; error 36 us; skew 500 ppm
Aug 06 18:36:46 dist-test-slave-1xrh krb5kdc[1592](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041406, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.194.61@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 18:36:46.614312 1685 init.cc:377] Logged in from keytab as kudu/127.19.194.61@KRBTEST.COM (short username kudu)
I20260806 18:36:46.615135 1685 webserver.cc:460] Webserver started at http://127.19.194.61:32821/ using document root <none> and password file <none>
I20260806 18:36:46.615684 1685 fs_manager.cc:359] Metadata directory not provided
I20260806 18:36:46.615774 1685 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 18:36:46.615994 1685 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 18:36:46.617947 1685 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786041246094603-20232-0/minicluster-data/master-1/data/instance:
uuid: "8fce36fcbb56469b9027d7bcfb145c91"
format_stamp: "Formatted at 2026-08-06 18:36:46 on dist-test-slave-1xrh"
server_key: "66374ef7e5db95f9aa9e9fd477065548"
server_key_iv: "8f3bb7647d09f3424e899c67dca53e98"
server_key_version: "encryptionkey@0"
I20260806 18:36:46.618520 1685 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786041246094603-20232-0/minicluster-data/master-1/wal/instance:
uuid: "8fce36fcbb56469b9027d7bcfb145c91"
format_stamp: "Formatted at 2026-08-06 18:36:46 on dist-test-slave-1xrh"
server_key: "66374ef7e5db95f9aa9e9fd477065548"
server_key_iv: "8f3bb7647d09f3424e899c67dca53e98"
server_key_version: "encryptionkey@0"
I20260806 18:36:46.622099 1685 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.006s sys 0.000s
I20260806 18:36:46.624888 1701 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 18:36:46.626036 1685 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260806 18:36:46.626179 1685 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786041246094603-20232-0/minicluster-data/master-1/data,/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786041246094603-20232-0/minicluster-data/master-1/wal
uuid: "8fce36fcbb56469b9027d7bcfb145c91"
format_stamp: "Formatted at 2026-08-06 18:36:46 on dist-test-slave-1xrh"
server_key: "66374ef7e5db95f9aa9e9fd477065548"
server_key_iv: "8f3bb7647d09f3424e899c67dca53e98"
server_key_version: "encryptionkey@0"
I20260806 18:36:46.626286 1685 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786041246094603-20232-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786041246094603-20232-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786041246094603-20232-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 18:36:46.641743 1685 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 18:36:46.644544 1685 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 18:36:46.644739 1685 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 18:36:46.651729 1685 rpc_server.cc:307] RPC server started. Bound to: 127.19.194.61:37625
I20260806 18:36:46.651741 1759 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.194.61:37625 every 8 connection(s)
I20260806 18:36:46.652667 1685 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786041246094603-20232-0/minicluster-data/master-1/data/info.pb
I20260806 18:36:46.655447 1760 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 18:36:46.655855 20232 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJW69tn/build/debug/bin/kudu as pid 1685
I20260806 18:36:46.655997 20232 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786041246094603-20232-0/minicluster-data/master-1/wal/instance
I20260806 18:36:46.656291 20232 external_mini_cluster.cc:1428] Setting key 4c1d64ddcff1bfd380b4b5fe5d2c7f62
I20260806 18:36:46.659924 1760 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.19.194.62" port: 35711 } has no permanent_uuid. Determining permanent_uuid...
WARNING: no policy specified for kudu/127.19.194.60@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.19.194.60@KRBTEST.COM" created.
Aug 06 18:36:46 dist-test-slave-1xrh krb5kdc[1592](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041406, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.194.61@KRBTEST.COM for kudu/127.19.194.62@KRBTEST.COM
I20260806 18:36:46.674170 1682 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:36:46.662392 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46286 (local address 127.19.194.62:35711)
0806 18:36:46.662532 (+ 140us) server_negotiation.cc:206] Beginning negotiation
0806 18:36:46.662536 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 18:36:46.663586 (+ 1050us) server_negotiation.cc:407] Connection header received
0806 18:36:46.664504 (+ 918us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:36:46.664508 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:36:46.664572 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:36:46.664674 (+ 102us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:36:46.665804 (+ 1130us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:36:46.666356 (+ 552us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:36:46.667581 (+ 1225us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:36:46.667865 (+ 284us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:36:46.667984 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:36:46.670564 (+ 2580us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:36:46.670568 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:36:46.670570 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 18:36:46.670593 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:36:46.672175 (+ 1582us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:36:46.672676 (+ 501us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:36:46.672681 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:36:46.672683 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:36:46.672759 (+ 76us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:36:46.673093 (+ 334us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:36:46.673098 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:36:46.673099 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:36:46.673274 (+ 175us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:36:46.673381 (+ 107us) server_negotiation.cc:1035] Waiting for connection context
0806 18:36:46.673893 (+ 512us) server_negotiation.cc:299] Negotiation successful
0806 18:36:46.674011 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260806 18:36:46.674880 1763 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:36:46.662899 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.194.62:35711 (local address 127.0.0.1:46286)
0806 18:36:46.663431 (+ 532us) negotiation.cc:107] Waiting for socket to connect
0806 18:36:46.663471 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0806 18:36:46.664284 (+ 813us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:36:46.664838 (+ 554us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:36:46.664845 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:36:46.665154 (+ 309us) client_negotiation.cc:189] Negotiated authn=SASL
0806 18:36:46.665656 (+ 502us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:36:46.665669 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:36:46.666501 (+ 832us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:36:46.666510 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:36:46.667450 (+ 940us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:36:46.667460 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:36:46.668057 (+ 597us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:36:46.668067 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:36:46.668240 (+ 173us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:36:46.668876 (+ 636us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 18:36:46.668893 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0806 18:36:46.670380 (+ 1487us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 18:36:46.672294 (+ 1914us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:36:46.672299 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:36:46.672320 (+ 21us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:36:46.672566 (+ 246us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:36:46.672859 (+ 293us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:36:46.672862 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:36:46.672864 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:36:46.672998 (+ 134us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:36:46.673382 (+ 384us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 18:36:46.673389 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 18:36:46.673618 (+ 229us) client_negotiation.cc:769] Sending connection context
0806 18:36:46.673795 (+ 177us) client_negotiation.cc:240] Negotiation successful
0806 18:36:46.674092 (+ 297us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":340,"thread_start_us":169,"threads_started":1}
I20260806 18:36:46.676267 1760 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.19.194.61" port: 37625 } has no permanent_uuid. Determining permanent_uuid...
Aug 06 18:36:46 dist-test-slave-1xrh krb5kdc[1592](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041406, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.194.61@KRBTEST.COM for kudu/127.19.194.61@KRBTEST.COM
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.19.194.60 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786041246094603-20232-0/krb5kdc/kudu_127.19.194.60.keytab.
Entry for principal kudu/127.19.194.60 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786041246094603-20232-0/krb5kdc/kudu_127.19.194.60.keytab.
I20260806 18:36:46.684396 20232 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJW69tn/build/debug/bin/kudu
/tmp/dist-test-taskJW69tn/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786041246094603-20232-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786041246094603-20232-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786041246094603-20232-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786041246094603-20232-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.19.194.60:46189
--webserver_interface=127.19.194.60
--webserver_port=0
--keytab_file=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786041246094603-20232-0/krb5kdc/kudu_127.19.194.60.keytab
--principal=kudu/127.19.194.60
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.19.194.20:38415
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.19.194.62:35711,127.19.194.61:37625,127.19.194.60:46189
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786041246094603-20232-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786041246094603-20232-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786041246094603-20232-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260806 18:36:46.685922 1678 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.19.194.61" port: 37625 } attempt: 3
I20260806 18:36:46.686535 1765 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:36:46.676929 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38594 (local address 127.19.194.61:37625)
0806 18:36:46.677182 (+ 253us) server_negotiation.cc:206] Beginning negotiation
0806 18:36:46.677190 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 18:36:46.677207 (+ 17us) server_negotiation.cc:407] Connection header received
0806 18:36:46.677236 (+ 29us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:36:46.677240 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:36:46.677370 (+ 130us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:36:46.677519 (+ 149us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:36:46.677818 (+ 299us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:36:46.678796 (+ 978us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:36:46.679811 (+ 1015us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:36:46.680312 (+ 501us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:36:46.680435 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:36:46.682463 (+ 2028us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:36:46.682472 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:36:46.682481 (+ 9us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 18:36:46.682501 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:36:46.684526 (+ 2025us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:36:46.685090 (+ 564us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:36:46.685099 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:36:46.685104 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:36:46.685194 (+ 90us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:36:46.685502 (+ 308us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:36:46.685508 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:36:46.685513 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:36:46.685843 (+ 330us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:36:46.685982 (+ 139us) server_negotiation.cc:1035] Waiting for connection context
0806 18:36:46.686231 (+ 249us) server_negotiation.cc:299] Negotiation successful
0806 18:36:46.686370 (+ 139us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":36,"server-negotiator.queue_time_us":177,"thread_start_us":90,"threads_started":1}
I20260806 18:36:46.687029 1763 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:36:46.676694 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.194.61:37625 (local address 127.0.0.1:38594)
0806 18:36:46.676815 (+ 121us) negotiation.cc:107] Waiting for socket to connect
0806 18:36:46.676830 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0806 18:36:46.676918 (+ 88us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:36:46.677499 (+ 581us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:36:46.677502 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:36:46.677636 (+ 134us) client_negotiation.cc:189] Negotiated authn=SASL
0806 18:36:46.677713 (+ 77us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:36:46.677717 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:36:46.678924 (+ 1207us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:36:46.678929 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:36:46.679687 (+ 758us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:36:46.679698 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:36:46.680438 (+ 740us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:36:46.680442 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:36:46.680627 (+ 185us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:36:46.681183 (+ 556us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 18:36:46.681202 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0806 18:36:46.682348 (+ 1146us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 18:36:46.684649 (+ 2301us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:36:46.684655 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:36:46.684660 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:36:46.684928 (+ 268us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:36:46.685306 (+ 378us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:36:46.685308 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:36:46.685310 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:36:46.685395 (+ 85us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:36:46.685946 (+ 551us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 18:36:46.685949 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 18:36:46.686029 (+ 80us) client_negotiation.cc:769] Sending connection context
0806 18:36:46.686147 (+ 118us) client_negotiation.cc:240] Negotiation successful
0806 18:36:46.686858 (+ 711us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":48,"mutex_wait_us":26}
I20260806 18:36:46.689138 1760 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.19.194.60" port: 46189 } has no permanent_uuid. Determining permanent_uuid...
I20260806 18:36:46.690137 1763 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:36:46.689583 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.194.60:46189 (local address 127.0.0.1:51578)
0806 18:36:46.689722 (+ 139us) negotiation.cc:107] Waiting for socket to connect
0806 18:36:46.690060 (+ 338us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.194.60:46189: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":53}
W20260806 18:36:46.690300 1705 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.19.194.60:46189: connect: Connection refused (error 111)
Aug 06 18:36:46 dist-test-slave-1xrh krb5kdc[1592](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041406, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.194.62@KRBTEST.COM for kudu/127.19.194.61@KRBTEST.COM
I20260806 18:36:46.691272 1769 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:36:46.690909 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.194.60:46189 (local address 127.0.0.1:51584)
0806 18:36:46.691141 (+ 232us) negotiation.cc:107] Waiting for socket to connect
0806 18:36:46.691189 (+ 48us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.194.60:46189: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":158,"thread_start_us":98,"threads_started":1}
W20260806 18:36:46.691403 1760 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.19.194.60:46189: Network error: Client connection negotiation failed: client connection to 127.19.194.60:46189: connect: Connection refused (error 111)
I20260806 18:36:46.695390 1765 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:36:46.686430 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38598 (local address 127.19.194.61:37625)
0806 18:36:46.686795 (+ 365us) server_negotiation.cc:206] Beginning negotiation
0806 18:36:46.686798 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 18:36:46.686810 (+ 12us) server_negotiation.cc:407] Connection header received
0806 18:36:46.686845 (+ 35us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:36:46.686847 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:36:46.686884 (+ 37us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:36:46.686959 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:36:46.687518 (+ 559us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:36:46.688179 (+ 661us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:36:46.689074 (+ 895us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:36:46.689377 (+ 303us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:36:46.689476 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:36:46.691705 (+ 2229us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:36:46.691708 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:36:46.691710 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 18:36:46.691729 (+ 19us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:36:46.693631 (+ 1902us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:36:46.694181 (+ 550us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:36:46.694184 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:36:46.694185 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:36:46.694254 (+ 69us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:36:46.694567 (+ 313us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:36:46.694570 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:36:46.694571 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:36:46.694718 (+ 147us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:36:46.694791 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0806 18:36:46.695205 (+ 414us) server_negotiation.cc:299] Negotiation successful
0806 18:36:46.695293 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":313,"thread_start_us":93,"threads_started":1}
I20260806 18:36:46.695519 1681 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:36:46.686364 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.194.61:37625 (local address 127.0.0.1:38598)
0806 18:36:46.686554 (+ 190us) negotiation.cc:107] Waiting for socket to connect
0806 18:36:46.686571 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0806 18:36:46.686664 (+ 93us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:36:46.687012 (+ 348us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:36:46.687016 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:36:46.687231 (+ 215us) client_negotiation.cc:189] Negotiated authn=SASL
0806 18:36:46.687379 (+ 148us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:36:46.687400 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:36:46.688293 (+ 893us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:36:46.688297 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:36:46.688922 (+ 625us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:36:46.688931 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:36:46.689481 (+ 550us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:36:46.689484 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:36:46.689587 (+ 103us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:36:46.690042 (+ 455us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 18:36:46.690057 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0806 18:36:46.691557 (+ 1500us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 18:36:46.693747 (+ 2190us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:36:46.693754 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:36:46.693757 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:36:46.694070 (+ 313us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:36:46.694347 (+ 277us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:36:46.694353 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:36:46.694358 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:36:46.694464 (+ 106us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:36:46.694825 (+ 361us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 18:36:46.694831 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 18:36:46.694974 (+ 143us) client_negotiation.cc:769] Sending connection context
0806 18:36:46.695105 (+ 131us) client_negotiation.cc:240] Negotiation successful
0806 18:36:46.695298 (+ 193us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":108}
I20260806 18:36:46.696439 1678 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.19.194.60" port: 46189 } has no permanent_uuid. Determining permanent_uuid...
I20260806 18:36:46.697048 1681 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:36:46.696806 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.194.60:46189 (local address 127.0.0.1:51598)
0806 18:36:46.696930 (+ 124us) negotiation.cc:107] Waiting for socket to connect
0806 18:36:46.696983 (+ 53us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.194.60:46189: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":48}
I20260806 18:36:46.697695 1681 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:36:46.697468 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.194.60:46189 (local address 127.0.0.1:51614)
0806 18:36:46.697608 (+ 140us) negotiation.cc:107] Waiting for socket to connect
0806 18:36:46.697649 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.194.60:46189: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":88}
W20260806 18:36:46.697865 1678 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.19.194.60:46189: Network error: Client connection negotiation failed: client connection to 127.19.194.60:46189: connect: Connection refused (error 111)
I20260806 18:36:46.729115 1678 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.19.194.60" port: 46189 } attempt: 1
I20260806 18:36:46.729974 1681 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:36:46.729665 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.194.60:46189 (local address 127.0.0.1:51630)
0806 18:36:46.729822 (+ 157us) negotiation.cc:107] Waiting for socket to connect
0806 18:36:46.729897 (+ 75us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.194.60:46189: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":53}
I20260806 18:36:46.730772 1681 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:36:46.730595 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.194.60:46189 (local address 127.0.0.1:51644)
0806 18:36:46.730681 (+ 86us) negotiation.cc:107] Waiting for socket to connect
0806 18:36:46.730724 (+ 43us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.194.60:46189: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":27}
W20260806 18:36:46.730937 1678 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.19.194.60:46189: Network error: Client connection negotiation failed: client connection to 127.19.194.60:46189: connect: Connection refused (error 111)
I20260806 18:36:46.740646 1760 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.19.194.60" port: 46189 } attempt: 1
I20260806 18:36:46.741333 1769 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:36:46.741079 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.194.60:46189 (local address 127.0.0.1:51652)
0806 18:36:46.741210 (+ 131us) negotiation.cc:107] Waiting for socket to connect
0806 18:36:46.741270 (+ 60us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.194.60:46189: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":57}
I20260806 18:36:46.742000 1769 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:36:46.741794 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.194.60:46189 (local address 127.0.0.1:51666)
0806 18:36:46.741896 (+ 102us) negotiation.cc:107] Waiting for socket to connect
0806 18:36:46.741940 (+ 44us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.194.60:46189: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":40}
W20260806 18:36:46.742120 1760 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.19.194.60:46189: Network error: Client connection negotiation failed: client connection to 127.19.194.60:46189: connect: Connection refused (error 111)
W20260806 18:36:46.795789 1766 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 18:36:46.796060 1766 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 18:36:46.796142 1766 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 18:36:46.799916 1766 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 18:36:46.799984 1766 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 18:36:46.800019 1766 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 18:36:46.800073 1766 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 18:36:46.800110 1766 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 18:36:46.804988 1766 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.194.20:38415
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786041246094603-20232-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786041246094603-20232-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.19.194.62:35711,127.19.194.61:37625,127.19.194.60:46189
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786041246094603-20232-0/krb5kdc/kudu_127.19.194.60.keytab
--principal=kudu/127.19.194.60
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.19.194.60:46189
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786041246094603-20232-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.19.194.60
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1766
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786041246094603-20232-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 17:59:13 UTC on ccycloud.quasar-wvsnzx.root.comops.site
build id 4571028
I20260806 18:36:46.805912 1766 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 18:36:46.806095 1678 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.19.194.60" port: 46189 } attempt: 2
I20260806 18:36:46.806879 1681 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:36:46.806609 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.194.60:46189 (local address 127.0.0.1:51674)
0806 18:36:46.806749 (+ 140us) negotiation.cc:107] Waiting for socket to connect
0806 18:36:46.806806 (+ 57us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.194.60:46189: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":53}
I20260806 18:36:46.807013 1766 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260806 18:36:46.807585 1681 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:36:46.807376 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.194.60:46189 (local address 127.0.0.1:51688)
0806 18:36:46.807479 (+ 103us) negotiation.cc:107] Waiting for socket to connect
0806 18:36:46.807529 (+ 50us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.194.60:46189: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":40}
W20260806 18:36:46.807722 1678 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.19.194.60:46189: Network error: Client connection negotiation failed: client connection to 127.19.194.60:46189: connect: Connection refused (error 111)
I20260806 18:36:46.810258 1760 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.19.194.60" port: 46189 } attempt: 2
I20260806 18:36:46.811021 1769 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:36:46.810672 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.194.60:46189 (local address 127.0.0.1:51704)
0806 18:36:46.810825 (+ 153us) negotiation.cc:107] Waiting for socket to connect
0806 18:36:46.810921 (+ 96us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.194.60:46189: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":62}
I20260806 18:36:46.811877 1769 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:36:46.811664 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.194.60:46189 (local address 127.0.0.1:51720)
0806 18:36:46.811773 (+ 109us) negotiation.cc:107] Waiting for socket to connect
0806 18:36:46.811819 (+ 46us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.194.60:46189: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":47}
W20260806 18:36:46.812032 1760 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.19.194.60:46189: Network error: Client connection negotiation failed: client connection to 127.19.194.60:46189: connect: Connection refused (error 111)
W20260806 18:36:46.815302 1775 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:36:46.815337 1776 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:36:46.815369 1778 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 18:36:46.815464 1766 server_base.cc:1034] running on GCE node
I20260806 18:36:46.816079 1766 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 18:36:46.817066 1766 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 18:36:46.818260 1766 hybrid_clock.cc:648] HybridClock initialized: now 1786041406818248 us; error 22 us; skew 500 ppm
Aug 06 18:36:46 dist-test-slave-1xrh krb5kdc[1592](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041406, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.194.60@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 18:36:46.820926 1766 init.cc:377] Logged in from keytab as kudu/127.19.194.60@KRBTEST.COM (short username kudu)
I20260806 18:36:46.821732 1766 webserver.cc:460] Webserver started at http://127.19.194.60:42041/ using document root <none> and password file <none>
I20260806 18:36:46.822288 1766 fs_manager.cc:359] Metadata directory not provided
I20260806 18:36:46.822355 1766 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 18:36:46.822513 1766 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 18:36:46.824585 1766 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786041246094603-20232-0/minicluster-data/master-2/data/instance:
uuid: "72b869b5e7574895bb11643ba521af41"
format_stamp: "Formatted at 2026-08-06 18:36:46 on dist-test-slave-1xrh"
server_key: "5dc7d3b3609a3bfab312909dc8486bbf"
server_key_iv: "b73c226ca0d935a4b5ab3b74ecf6f790"
server_key_version: "encryptionkey@0"
I20260806 18:36:46.825074 1766 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786041246094603-20232-0/minicluster-data/master-2/wal/instance:
uuid: "72b869b5e7574895bb11643ba521af41"
format_stamp: "Formatted at 2026-08-06 18:36:46 on dist-test-slave-1xrh"
server_key: "5dc7d3b3609a3bfab312909dc8486bbf"
server_key_iv: "b73c226ca0d935a4b5ab3b74ecf6f790"
server_key_version: "encryptionkey@0"
I20260806 18:36:46.828526 1766 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20260806 18:36:46.830926 1786 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 18:36:46.832127 1766 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260806 18:36:46.832222 1766 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786041246094603-20232-0/minicluster-data/master-2/data,/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786041246094603-20232-0/minicluster-data/master-2/wal
uuid: "72b869b5e7574895bb11643ba521af41"
format_stamp: "Formatted at 2026-08-06 18:36:46 on dist-test-slave-1xrh"
server_key: "5dc7d3b3609a3bfab312909dc8486bbf"
server_key_iv: "b73c226ca0d935a4b5ab3b74ecf6f790"
server_key_version: "encryptionkey@0"
I20260806 18:36:46.832309 1766 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786041246094603-20232-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786041246094603-20232-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786041246094603-20232-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 18:36:46.847936 1766 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 18:36:46.850368 1766 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 18:36:46.850519 1766 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 18:36:46.857084 1766 rpc_server.cc:307] RPC server started. Bound to: 127.19.194.60:46189
I20260806 18:36:46.857121 1841 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.194.60:46189 every 8 connection(s)
I20260806 18:36:46.857908 1766 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786041246094603-20232-0/minicluster-data/master-2/data/info.pb
I20260806 18:36:46.860394 1842 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 18:36:46.861227 20232 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJW69tn/build/debug/bin/kudu as pid 1766
I20260806 18:36:46.861311 20232 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786041246094603-20232-0/minicluster-data/master-2/wal/instance
I20260806 18:36:46.861584 20232 external_mini_cluster.cc:1428] Setting key 77edf9994ab011d09938bab7e2624195
I20260806 18:36:46.864362 1842 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.19.194.62" port: 35711 } has no permanent_uuid. Determining permanent_uuid...
Aug 06 18:36:46 dist-test-slave-1xrh krb5kdc[1592](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041406, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.19.194.60@KRBTEST.COM
Aug 06 18:36:46 dist-test-slave-1xrh krb5kdc[1592](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041406, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.194.60@KRBTEST.COM for kudu/127.19.194.62@KRBTEST.COM
I20260806 18:36:46.876031 1682 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:36:46.865993 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46296 (local address 127.19.194.62:35711)
0806 18:36:46.866137 (+ 144us) server_negotiation.cc:206] Beginning negotiation
0806 18:36:46.866142 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 18:36:46.866409 (+ 267us) server_negotiation.cc:407] Connection header received
0806 18:36:46.866538 (+ 129us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:36:46.866541 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:36:46.866595 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:36:46.866685 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:36:46.867523 (+ 838us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:36:46.868188 (+ 665us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:36:46.869321 (+ 1133us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:36:46.869525 (+ 204us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:36:46.869596 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:36:46.872060 (+ 2464us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:36:46.872064 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:36:46.872068 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 18:36:46.872103 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:36:46.874235 (+ 2132us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:36:46.874660 (+ 425us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:36:46.874664 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:36:46.874667 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:36:46.874757 (+ 90us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:36:46.875097 (+ 340us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:36:46.875100 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:36:46.875102 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:36:46.875330 (+ 228us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:36:46.875431 (+ 101us) server_negotiation.cc:1035] Waiting for connection context
0806 18:36:46.875772 (+ 341us) server_negotiation.cc:299] Negotiation successful
0806 18:36:46.875894 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260806 18:36:46.876273 1845 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:36:46.862840 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51732 (local address 127.19.194.60:46189)
0806 18:36:46.863298 (+ 458us) server_negotiation.cc:206] Beginning negotiation
0806 18:36:46.863330 (+ 32us) server_negotiation.cc:399] Waiting for connection header
0806 18:36:46.863364 (+ 34us) server_negotiation.cc:407] Connection header received
0806 18:36:46.864022 (+ 658us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:36:46.864039 (+ 17us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:36:46.864332 (+ 293us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:36:46.864650 (+ 318us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:36:46.865459 (+ 809us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:36:46.866664 (+ 1205us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:36:46.867792 (+ 1128us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:36:46.868445 (+ 653us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:36:46.868573 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:36:46.871227 (+ 2654us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:36:46.871233 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:36:46.871241 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 18:36:46.871262 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:36:46.873313 (+ 2051us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:36:46.873770 (+ 457us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:36:46.873776 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:36:46.873782 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:36:46.873915 (+ 133us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:36:46.874218 (+ 303us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:36:46.874221 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:36:46.874223 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:36:46.874639 (+ 416us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:36:46.874845 (+ 206us) server_negotiation.cc:1035] Waiting for connection context
0806 18:36:46.875125 (+ 280us) server_negotiation.cc:299] Negotiation successful
0806 18:36:46.875428 (+ 303us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":337,"thread_start_us":126,"threads_started":1}
I20260806 18:36:46.876277 1846 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:36:46.865893 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.194.62:35711 (local address 127.0.0.1:46296)
0806 18:36:46.866280 (+ 387us) negotiation.cc:107] Waiting for socket to connect
0806 18:36:46.866325 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0806 18:36:46.866444 (+ 119us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:36:46.866755 (+ 311us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:36:46.866762 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:36:46.867226 (+ 464us) client_negotiation.cc:189] Negotiated authn=SASL
0806 18:36:46.867389 (+ 163us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:36:46.867396 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:36:46.868307 (+ 911us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:36:46.868310 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:36:46.869190 (+ 880us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:36:46.869202 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:36:46.869680 (+ 478us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:36:46.869688 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:36:46.869822 (+ 134us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:36:46.870335 (+ 513us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 18:36:46.870349 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0806 18:36:46.871904 (+ 1555us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 18:36:46.874331 (+ 2427us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:36:46.874339 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:36:46.874343 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:36:46.874559 (+ 216us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:36:46.874886 (+ 327us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:36:46.874890 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:36:46.874893 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:36:46.874997 (+ 104us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:36:46.875465 (+ 468us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 18:36:46.875470 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 18:36:46.875563 (+ 93us) client_negotiation.cc:769] Sending connection context
0806 18:36:46.875673 (+ 110us) client_negotiation.cc:240] Negotiation successful
0806 18:36:46.875794 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":235,"thread_start_us":117,"threads_started":1}
I20260806 18:36:46.877629 1842 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.19.194.61" port: 37625 } has no permanent_uuid. Determining permanent_uuid...
Aug 06 18:36:46 dist-test-slave-1xrh krb5kdc[1592](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041406, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.194.60@KRBTEST.COM for kudu/127.19.194.61@KRBTEST.COM
Aug 06 18:36:46 dist-test-slave-1xrh krb5kdc[1592](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041406, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.19.194.62@KRBTEST.COM
I20260806 18:36:46.885746 1846 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:36:46.878095 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.194.61:37625 (local address 127.0.0.1:38600)
0806 18:36:46.878280 (+ 185us) negotiation.cc:107] Waiting for socket to connect
0806 18:36:46.878294 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0806 18:36:46.878369 (+ 75us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:36:46.878615 (+ 246us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:36:46.878618 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:36:46.878771 (+ 153us) client_negotiation.cc:189] Negotiated authn=SASL
0806 18:36:46.878910 (+ 139us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:36:46.878916 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:36:46.879692 (+ 776us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:36:46.879694 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:36:46.880216 (+ 522us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:36:46.880244 (+ 28us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:36:46.880669 (+ 425us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:36:46.880672 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:36:46.880763 (+ 91us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:36:46.881133 (+ 370us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 18:36:46.881144 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0806 18:36:46.882382 (+ 1238us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 18:36:46.884271 (+ 1889us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:36:46.884275 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:36:46.884276 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:36:46.884511 (+ 235us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:36:46.884807 (+ 296us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:36:46.884810 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:36:46.884812 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:36:46.884904 (+ 92us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:36:46.885271 (+ 367us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 18:36:46.885274 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 18:36:46.885369 (+ 95us) client_negotiation.cc:769] Sending connection context
0806 18:36:46.885475 (+ 106us) client_negotiation.cc:240] Negotiation successful
0806 18:36:46.885596 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":112}
I20260806 18:36:46.885788 1765 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:36:46.878241 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38600 (local address 127.19.194.61:37625)
0806 18:36:46.878350 (+ 109us) server_negotiation.cc:206] Beginning negotiation
0806 18:36:46.878356 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 18:36:46.878394 (+ 38us) server_negotiation.cc:407] Connection header received
0806 18:36:46.878443 (+ 49us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:36:46.878445 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:36:46.878496 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:36:46.878583 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:36:46.879032 (+ 449us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:36:46.879598 (+ 566us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:36:46.880350 (+ 752us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:36:46.880566 (+ 216us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:36:46.880638 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:36:46.882543 (+ 1905us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:36:46.882547 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:36:46.882551 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 18:36:46.882576 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:36:46.884162 (+ 1586us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:36:46.884623 (+ 461us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:36:46.884632 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:36:46.884634 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:36:46.884711 (+ 77us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:36:46.884998 (+ 287us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:36:46.885001 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:36:46.885003 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:36:46.885171 (+ 168us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:36:46.885276 (+ 105us) server_negotiation.cc:1035] Waiting for connection context
0806 18:36:46.885542 (+ 266us) server_negotiation.cc:299] Negotiation successful
0806 18:36:46.885652 (+ 110us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":33}
I20260806 18:36:46.886566 1842 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.19.194.60" port: 46189 } has no permanent_uuid. Determining permanent_uuid...
I20260806 18:36:46.887764 1682 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:36:46.878532 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46300 (local address 127.19.194.62:35711)
0806 18:36:46.878658 (+ 126us) server_negotiation.cc:206] Beginning negotiation
0806 18:36:46.878663 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 18:36:46.878807 (+ 144us) server_negotiation.cc:407] Connection header received
0806 18:36:46.878869 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:36:46.878874 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:36:46.878917 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:36:46.878996 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:36:46.880256 (+ 1260us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:36:46.880858 (+ 602us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:36:46.881898 (+ 1040us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:36:46.882120 (+ 222us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:36:46.882221 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:36:46.884399 (+ 2178us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:36:46.884402 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:36:46.884404 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 18:36:46.884424 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:36:46.885910 (+ 1486us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:36:46.886424 (+ 514us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:36:46.886428 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:36:46.886430 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:36:46.886544 (+ 114us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:36:46.886880 (+ 336us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:36:46.886882 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:36:46.886883 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:36:46.887105 (+ 222us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:36:46.887363 (+ 258us) server_negotiation.cc:1035] Waiting for connection context
0806 18:36:46.887501 (+ 138us) server_negotiation.cc:299] Negotiation successful
0806 18:36:46.887620 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
Aug 06 18:36:46 dist-test-slave-1xrh krb5kdc[1592](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041406, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.194.60@KRBTEST.COM for kudu/127.19.194.60@KRBTEST.COM
Aug 06 18:36:46 dist-test-slave-1xrh krb5kdc[1592](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041406, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.19.194.61@KRBTEST.COM
I20260806 18:36:46.895498 1845 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:36:46.887083 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51738 (local address 127.19.194.60:46189)
0806 18:36:46.887195 (+ 112us) server_negotiation.cc:206] Beginning negotiation
0806 18:36:46.887200 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 18:36:46.887212 (+ 12us) server_negotiation.cc:407] Connection header received
0806 18:36:46.887300 (+ 88us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:36:46.887303 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:36:46.887351 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:36:46.887427 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:36:46.887804 (+ 377us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:36:46.888679 (+ 875us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:36:46.890133 (+ 1454us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:36:46.890346 (+ 213us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:36:46.890420 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:36:46.892488 (+ 2068us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:36:46.892491 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:36:46.892493 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 18:36:46.892513 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:36:46.893945 (+ 1432us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:36:46.894320 (+ 375us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:36:46.894325 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:36:46.894327 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:36:46.894398 (+ 71us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:36:46.894655 (+ 257us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:36:46.894659 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:36:46.894661 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:36:46.894806 (+ 145us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:36:46.894964 (+ 158us) server_negotiation.cc:1035] Waiting for connection context
0806 18:36:46.895233 (+ 269us) server_negotiation.cc:299] Negotiation successful
0806 18:36:46.895370 (+ 137us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":38,"server-negotiator.queue_time_us":54}
I20260806 18:36:46.895483 1846 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:36:46.887033 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.194.60:46189 (local address 127.0.0.1:51738)
0806 18:36:46.887150 (+ 117us) negotiation.cc:107] Waiting for socket to connect
0806 18:36:46.887165 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0806 18:36:46.887231 (+ 66us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:36:46.887455 (+ 224us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:36:46.887458 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:36:46.887621 (+ 163us) client_negotiation.cc:189] Negotiated authn=SASL
0806 18:36:46.887701 (+ 80us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:36:46.887707 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:36:46.889202 (+ 1495us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:36:46.889207 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:36:46.890012 (+ 805us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:36:46.890023 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:36:46.890450 (+ 427us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:36:46.890455 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:36:46.890549 (+ 94us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:36:46.890960 (+ 411us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 18:36:46.890971 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0806 18:36:46.892354 (+ 1383us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 18:36:46.894046 (+ 1692us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:36:46.894048 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:36:46.894049 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:36:46.894222 (+ 173us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:36:46.894493 (+ 271us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:36:46.894495 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:36:46.894497 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:36:46.894569 (+ 72us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:36:46.894957 (+ 388us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 18:36:46.894959 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 18:36:46.895043 (+ 84us) client_negotiation.cc:769] Sending connection context
0806 18:36:46.895187 (+ 144us) client_negotiation.cc:240] Negotiation successful
0806 18:36:46.895316 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":26}
I20260806 18:36:46.897207 1765 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:36:46.888825 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38608 (local address 127.19.194.61:37625)
0806 18:36:46.888916 (+ 91us) server_negotiation.cc:206] Beginning negotiation
0806 18:36:46.888919 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 18:36:46.888994 (+ 75us) server_negotiation.cc:407] Connection header received
0806 18:36:46.889172 (+ 178us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:36:46.889175 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:36:46.889217 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:36:46.889289 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:36:46.889827 (+ 538us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:36:46.890430 (+ 603us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:36:46.891389 (+ 959us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:36:46.891676 (+ 287us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:36:46.891754 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:36:46.893985 (+ 2231us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:36:46.893990 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:36:46.893994 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 18:36:46.894024 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:36:46.895528 (+ 1504us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:36:46.896077 (+ 549us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:36:46.896082 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:36:46.896083 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:36:46.896166 (+ 83us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:36:46.896504 (+ 338us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:36:46.896507 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:36:46.896508 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:36:46.896665 (+ 157us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:36:46.896845 (+ 180us) server_negotiation.cc:1035] Waiting for connection context
0806 18:36:46.897025 (+ 180us) server_negotiation.cc:299] Negotiation successful
0806 18:36:46.897107 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
I20260806 18:36:46.898942 1842 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 72b869b5e7574895bb11643ba521af41: Bootstrap starting.
I20260806 18:36:46.900375 20232 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260806 18:36:46.901526 1842 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 72b869b5e7574895bb11643ba521af41: Neither blocks nor log segments found. Creating new log.
I20260806 18:36:46.902621 1842 log.cc:824] T 00000000000000000000000000000000 P 72b869b5e7574895bb11643ba521af41: Log is configured to *not* fsync() on all Append() calls
I20260806 18:36:46.903215 1760 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.19.194.60" port: 46189 } attempt: 3
I20260806 18:36:46.905009 1842 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 72b869b5e7574895bb11643ba521af41: No bootstrap required, opened a new log
I20260806 18:36:46.906874 1678 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.19.194.60" port: 46189 } attempt: 3
I20260806 18:36:46.910511 1842 raft_consensus.cc:348] T 00000000000000000000000000000000 P 72b869b5e7574895bb11643ba521af41 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "168bc8903141442bab766da1d878ea87" member_type: VOTER last_known_addr { host: "127.19.194.62" port: 35711 } } peers { permanent_uuid: "8fce36fcbb56469b9027d7bcfb145c91" member_type: VOTER last_known_addr { host: "127.19.194.61" port: 37625 } } peers { permanent_uuid: "72b869b5e7574895bb11643ba521af41" member_type: VOTER last_known_addr { host: "127.19.194.60" port: 46189 } }
I20260806 18:36:46.910864 1842 raft_consensus.cc:374] T 00000000000000000000000000000000 P 72b869b5e7574895bb11643ba521af41 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 18:36:46.910961 1842 raft_consensus.cc:729] T 00000000000000000000000000000000 P 72b869b5e7574895bb11643ba521af41 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 72b869b5e7574895bb11643ba521af41, State: Initialized, Role: FOLLOWER
I20260806 18:36:46.911548 1842 consensus_queue.cc:260] T 00000000000000000000000000000000 P 72b869b5e7574895bb11643ba521af41 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "168bc8903141442bab766da1d878ea87" member_type: VOTER last_known_addr { host: "127.19.194.62" port: 35711 } } peers { permanent_uuid: "8fce36fcbb56469b9027d7bcfb145c91" member_type: VOTER last_known_addr { host: "127.19.194.61" port: 37625 } } peers { permanent_uuid: "72b869b5e7574895bb11643ba521af41" member_type: VOTER last_known_addr { host: "127.19.194.60" port: 46189 } }
I20260806 18:36:46.913074 1842 sys_catalog.cc:564] T 00000000000000000000000000000000 P 72b869b5e7574895bb11643ba521af41 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 18:36:46.913206 1765 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:36:46.902398 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38616 (local address 127.19.194.61:37625)
0806 18:36:46.902513 (+ 115us) server_negotiation.cc:206] Beginning negotiation
0806 18:36:46.902517 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 18:36:46.903183 (+ 666us) server_negotiation.cc:407] Connection header received
0806 18:36:46.903320 (+ 137us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:36:46.903323 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:36:46.903383 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:36:46.903438 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:36:46.905153 (+ 1715us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:36:46.905835 (+ 682us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:36:46.907009 (+ 1174us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:36:46.907365 (+ 356us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:36:46.907450 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:36:46.909376 (+ 1926us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:36:46.909383 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:36:46.909388 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 18:36:46.909414 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:36:46.911275 (+ 1861us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:36:46.911771 (+ 496us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:36:46.911774 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:36:46.911777 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:36:46.911873 (+ 96us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:36:46.912207 (+ 334us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:36:46.912211 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:36:46.912214 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:36:46.912458 (+ 244us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:36:46.912559 (+ 101us) server_negotiation.cc:1035] Waiting for connection context
0806 18:36:46.912920 (+ 361us) server_negotiation.cc:299] Negotiation successful
0806 18:36:46.913031 (+ 111us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59,"spinlock_wait_cycles":11648}
Aug 06 18:36:46 dist-test-slave-1xrh krb5kdc[1592](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041406, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.194.61@KRBTEST.COM for kudu/127.19.194.60@KRBTEST.COM
I20260806 18:36:46.913772 1845 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:36:46.902781 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51740 (local address 127.19.194.60:46189)
0806 18:36:46.902877 (+ 96us) server_negotiation.cc:206] Beginning negotiation
0806 18:36:46.902881 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 18:36:46.903535 (+ 654us) server_negotiation.cc:407] Connection header received
0806 18:36:46.903645 (+ 110us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:36:46.903649 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:36:46.903692 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:36:46.903768 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:36:46.904820 (+ 1052us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:36:46.905583 (+ 763us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:36:46.906917 (+ 1334us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:36:46.907255 (+ 338us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:36:46.907331 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:36:46.909420 (+ 2089us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:36:46.909424 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:36:46.909426 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 18:36:46.909451 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:36:46.911617 (+ 2166us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:36:46.912366 (+ 749us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:36:46.912370 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:36:46.912372 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:36:46.912451 (+ 79us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:36:46.912781 (+ 330us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:36:46.912788 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:36:46.912792 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:36:46.913027 (+ 235us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:36:46.913122 (+ 95us) server_negotiation.cc:1035] Waiting for connection context
0806 18:36:46.913510 (+ 388us) server_negotiation.cc:299] Negotiation successful
0806 18:36:46.913609 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260806 18:36:46.918445 1769 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:36:46.903683 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.194.60:46189 (local address 127.0.0.1:51744)
0806 18:36:46.903903 (+ 220us) negotiation.cc:107] Waiting for socket to connect
0806 18:36:46.903916 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0806 18:36:46.903994 (+ 78us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:36:46.908839 (+ 4845us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:36:46.908842 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:36:46.909032 (+ 190us) client_negotiation.cc:189] Negotiated authn=SASL
0806 18:36:46.909155 (+ 123us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:36:46.909161 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:36:46.910273 (+ 1112us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:36:46.910276 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:36:46.911133 (+ 857us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:36:46.911169 (+ 36us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:36:46.911771 (+ 602us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:36:46.911774 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:36:46.911870 (+ 96us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:36:46.912314 (+ 444us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 18:36:46.912341 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0806 18:36:46.913792 (+ 1451us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 18:36:46.916824 (+ 3032us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:36:46.916828 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:36:46.916830 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:36:46.917076 (+ 246us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:36:46.917408 (+ 332us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:36:46.917411 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:36:46.917413 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:36:46.917511 (+ 98us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:36:46.917961 (+ 450us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 18:36:46.917964 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 18:36:46.918056 (+ 92us) client_negotiation.cc:769] Sending connection context
0806 18:36:46.918140 (+ 84us) client_negotiation.cc:240] Negotiation successful
0806 18:36:46.918270 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":159}
I20260806 18:36:46.919014 1855 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:36:46.904113 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51744 (local address 127.19.194.60:46189)
0806 18:36:46.908137 (+ 4024us) server_negotiation.cc:206] Beginning negotiation
0806 18:36:46.908593 (+ 456us) server_negotiation.cc:399] Waiting for connection header
0806 18:36:46.908615 (+ 22us) server_negotiation.cc:407] Connection header received
0806 18:36:46.908648 (+ 33us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:36:46.908650 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:36:46.908701 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:36:46.909267 (+ 566us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:36:46.909396 (+ 129us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:36:46.910125 (+ 729us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:36:46.911302 (+ 1177us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:36:46.911632 (+ 330us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:36:46.912077 (+ 445us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:36:46.914144 (+ 2067us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:36:46.914147 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:36:46.914149 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 18:36:46.914171 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:36:46.916706 (+ 2535us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:36:46.917189 (+ 483us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:36:46.917192 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:36:46.917193 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:36:46.917277 (+ 84us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:36:46.917631 (+ 354us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:36:46.917634 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:36:46.917636 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:36:46.917841 (+ 205us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:36:46.918614 (+ 773us) server_negotiation.cc:1035] Waiting for connection context
0806 18:36:46.918714 (+ 100us) server_negotiation.cc:299] Negotiation successful
0806 18:36:46.918823 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":3920,"thread_start_us":90,"threads_started":1}
I20260806 18:36:46.919317 1682 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:36:46.902338 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46312 (local address 127.19.194.62:35711)
0806 18:36:46.902460 (+ 122us) server_negotiation.cc:206] Beginning negotiation
0806 18:36:46.902466 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 18:36:46.903276 (+ 810us) server_negotiation.cc:407] Connection header received
0806 18:36:46.903807 (+ 531us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:36:46.903811 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:36:46.903860 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:36:46.903915 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:36:46.904864 (+ 949us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:36:46.905583 (+ 719us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:36:46.907202 (+ 1619us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:36:46.907515 (+ 313us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:36:46.907587 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:36:46.915341 (+ 7754us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:36:46.915345 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:36:46.915347 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 18:36:46.915376 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:36:46.917038 (+ 1662us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:36:46.917550 (+ 512us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:36:46.917556 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:36:46.917558 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:36:46.917640 (+ 82us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:36:46.917958 (+ 318us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:36:46.917961 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:36:46.917963 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:36:46.918182 (+ 219us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:36:46.918291 (+ 109us) server_negotiation.cc:1035] Waiting for connection context
0806 18:36:46.919013 (+ 722us) server_negotiation.cc:299] Negotiation successful
0806 18:36:46.919158 (+ 145us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
Aug 06 18:36:46 dist-test-slave-1xrh krb5kdc[1592](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041406, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.194.62@KRBTEST.COM for kudu/127.19.194.60@KRBTEST.COM
I20260806 18:36:46.921829 1857 sys_catalog.cc:455] T 00000000000000000000000000000000 P 72b869b5e7574895bb11643ba521af41 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "168bc8903141442bab766da1d878ea87" member_type: VOTER last_known_addr { host: "127.19.194.62" port: 35711 } } peers { permanent_uuid: "8fce36fcbb56469b9027d7bcfb145c91" member_type: VOTER last_known_addr { host: "127.19.194.61" port: 37625 } } peers { permanent_uuid: "72b869b5e7574895bb11643ba521af41" member_type: VOTER last_known_addr { host: "127.19.194.60" port: 46189 } } }
I20260806 18:36:46.921965 1857 sys_catalog.cc:458] T 00000000000000000000000000000000 P 72b869b5e7574895bb11643ba521af41 [sys.catalog]: This master's current role is: FOLLOWER
I20260806 18:36:46.917546 1766 master_runner.cc:186] Error getting master registration for 127.19.194.62:35711: OK, instance_id { permanent_uuid: "168bc8903141442bab766da1d878ea87" instance_seqno: 1786041406441474 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260806 18:36:46.922204 1760 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8fce36fcbb56469b9027d7bcfb145c91: Bootstrap starting.
I20260806 18:36:46.922397 1766 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
W20260806 18:36:46.924861 1869 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 72b869b5e7574895bb11643ba521af41: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260806 18:36:46.924934 1869 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260806 18:36:46.925451 1760 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 8fce36fcbb56469b9027d7bcfb145c91: Neither blocks nor log segments found. Creating new log.
I20260806 18:36:46.925796 1681 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:36:46.907277 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.194.60:46189 (local address 127.0.0.1:51760)
0806 18:36:46.907497 (+ 220us) negotiation.cc:107] Waiting for socket to connect
0806 18:36:46.907514 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0806 18:36:46.907581 (+ 67us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:36:46.914699 (+ 7118us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:36:46.914706 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:36:46.914892 (+ 186us) client_negotiation.cc:189] Negotiated authn=SASL
0806 18:36:46.914999 (+ 107us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:36:46.915007 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:36:46.917771 (+ 2764us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:36:46.917774 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:36:46.918655 (+ 881us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:36:46.918667 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:36:46.919733 (+ 1066us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:36:46.919737 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:36:46.919820 (+ 83us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:36:46.920248 (+ 428us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 18:36:46.920265 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0806 18:36:46.921627 (+ 1362us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 18:36:46.924397 (+ 2770us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:36:46.924400 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:36:46.924402 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:36:46.924595 (+ 193us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:36:46.924891 (+ 296us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:36:46.924894 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:36:46.924896 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:36:46.924993 (+ 97us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:36:46.925365 (+ 372us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 18:36:46.925370 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 18:36:46.925461 (+ 91us) client_negotiation.cc:769] Sending connection context
0806 18:36:46.925569 (+ 108us) client_negotiation.cc:240] Negotiation successful
0806 18:36:46.925691 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":156}
I20260806 18:36:46.926342 1856 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:36:46.907711 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51760 (local address 127.19.194.60:46189)
0806 18:36:46.914038 (+ 6327us) server_negotiation.cc:206] Beginning negotiation
0806 18:36:46.914049 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0806 18:36:46.914059 (+ 10us) server_negotiation.cc:407] Connection header received
0806 18:36:46.914087 (+ 28us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:36:46.914445 (+ 358us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:36:46.914506 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:36:46.915777 (+ 1271us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:36:46.915978 (+ 201us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:36:46.917644 (+ 1666us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:36:46.918911 (+ 1267us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:36:46.919598 (+ 687us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:36:46.920016 (+ 418us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:36:46.922068 (+ 2052us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:36:46.922071 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:36:46.922074 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 18:36:46.922099 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:36:46.924276 (+ 2177us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:36:46.924693 (+ 417us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:36:46.924697 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:36:46.924699 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:36:46.924785 (+ 86us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:36:46.925075 (+ 290us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:36:46.925079 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:36:46.925080 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:36:46.925249 (+ 169us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:36:46.925350 (+ 101us) server_negotiation.cc:1035] Waiting for connection context
0806 18:36:46.926121 (+ 771us) server_negotiation.cc:299] Negotiation successful
0806 18:36:46.926211 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":4509,"thread_start_us":312,"threads_started":1}
I20260806 18:36:46.926343 1760 log.cc:824] T 00000000000000000000000000000000 P 8fce36fcbb56469b9027d7bcfb145c91: Log is configured to *not* fsync() on all Append() calls
I20260806 18:36:46.928872 1678 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 168bc8903141442bab766da1d878ea87: Bootstrap starting.
I20260806 18:36:46.928908 1760 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8fce36fcbb56469b9027d7bcfb145c91: No bootstrap required, opened a new log
I20260806 18:36:46.931208 1678 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 168bc8903141442bab766da1d878ea87: Neither blocks nor log segments found. Creating new log.
I20260806 18:36:46.931879 1678 log.cc:824] T 00000000000000000000000000000000 P 168bc8903141442bab766da1d878ea87: Log is configured to *not* fsync() on all Append() calls
I20260806 18:36:46.933072 1760 raft_consensus.cc:348] T 00000000000000000000000000000000 P 8fce36fcbb56469b9027d7bcfb145c91 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "168bc8903141442bab766da1d878ea87" member_type: VOTER last_known_addr { host: "127.19.194.62" port: 35711 } } peers { permanent_uuid: "8fce36fcbb56469b9027d7bcfb145c91" member_type: VOTER last_known_addr { host: "127.19.194.61" port: 37625 } } peers { permanent_uuid: "72b869b5e7574895bb11643ba521af41" member_type: VOTER last_known_addr { host: "127.19.194.60" port: 46189 } }
I20260806 18:36:46.933300 1760 raft_consensus.cc:374] T 00000000000000000000000000000000 P 8fce36fcbb56469b9027d7bcfb145c91 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 18:36:46.933362 1760 raft_consensus.cc:729] T 00000000000000000000000000000000 P 8fce36fcbb56469b9027d7bcfb145c91 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8fce36fcbb56469b9027d7bcfb145c91, State: Initialized, Role: FOLLOWER
I20260806 18:36:46.933852 1760 consensus_queue.cc:260] T 00000000000000000000000000000000 P 8fce36fcbb56469b9027d7bcfb145c91 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "168bc8903141442bab766da1d878ea87" member_type: VOTER last_known_addr { host: "127.19.194.62" port: 35711 } } peers { permanent_uuid: "8fce36fcbb56469b9027d7bcfb145c91" member_type: VOTER last_known_addr { host: "127.19.194.61" port: 37625 } } peers { permanent_uuid: "72b869b5e7574895bb11643ba521af41" member_type: VOTER last_known_addr { host: "127.19.194.60" port: 46189 } }
I20260806 18:36:46.934268 1678 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 168bc8903141442bab766da1d878ea87: No bootstrap required, opened a new log
I20260806 18:36:46.934414 1874 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8fce36fcbb56469b9027d7bcfb145c91 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "168bc8903141442bab766da1d878ea87" member_type: VOTER last_known_addr { host: "127.19.194.62" port: 35711 } } peers { permanent_uuid: "8fce36fcbb56469b9027d7bcfb145c91" member_type: VOTER last_known_addr { host: "127.19.194.61" port: 37625 } } peers { permanent_uuid: "72b869b5e7574895bb11643ba521af41" member_type: VOTER last_known_addr { host: "127.19.194.60" port: 46189 } } }
I20260806 18:36:46.934620 1874 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8fce36fcbb56469b9027d7bcfb145c91 [sys.catalog]: This master's current role is: FOLLOWER
I20260806 18:36:46.935205 1760 sys_catalog.cc:564] T 00000000000000000000000000000000 P 8fce36fcbb56469b9027d7bcfb145c91 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 18:36:46.939219 1678 raft_consensus.cc:348] T 00000000000000000000000000000000 P 168bc8903141442bab766da1d878ea87 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "168bc8903141442bab766da1d878ea87" member_type: VOTER last_known_addr { host: "127.19.194.62" port: 35711 } } peers { permanent_uuid: "8fce36fcbb56469b9027d7bcfb145c91" member_type: VOTER last_known_addr { host: "127.19.194.61" port: 37625 } } peers { permanent_uuid: "72b869b5e7574895bb11643ba521af41" member_type: VOTER last_known_addr { host: "127.19.194.60" port: 46189 } }
I20260806 18:36:46.939420 1678 raft_consensus.cc:374] T 00000000000000000000000000000000 P 168bc8903141442bab766da1d878ea87 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 18:36:46.939466 1678 raft_consensus.cc:729] T 00000000000000000000000000000000 P 168bc8903141442bab766da1d878ea87 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 168bc8903141442bab766da1d878ea87, State: Initialized, Role: FOLLOWER
I20260806 18:36:46.939934 1678 consensus_queue.cc:260] T 00000000000000000000000000000000 P 168bc8903141442bab766da1d878ea87 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "168bc8903141442bab766da1d878ea87" member_type: VOTER last_known_addr { host: "127.19.194.62" port: 35711 } } peers { permanent_uuid: "8fce36fcbb56469b9027d7bcfb145c91" member_type: VOTER last_known_addr { host: "127.19.194.61" port: 37625 } } peers { permanent_uuid: "72b869b5e7574895bb11643ba521af41" member_type: VOTER last_known_addr { host: "127.19.194.60" port: 46189 } }
I20260806 18:36:46.940430 1887 sys_catalog.cc:455] T 00000000000000000000000000000000 P 168bc8903141442bab766da1d878ea87 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "168bc8903141442bab766da1d878ea87" member_type: VOTER last_known_addr { host: "127.19.194.62" port: 35711 } } peers { permanent_uuid: "8fce36fcbb56469b9027d7bcfb145c91" member_type: VOTER last_known_addr { host: "127.19.194.61" port: 37625 } } peers { permanent_uuid: "72b869b5e7574895bb11643ba521af41" member_type: VOTER last_known_addr { host: "127.19.194.60" port: 46189 } } }
I20260806 18:36:46.940554 1887 sys_catalog.cc:458] T 00000000000000000000000000000000 P 168bc8903141442bab766da1d878ea87 [sys.catalog]: This master's current role is: FOLLOWER
I20260806 18:36:46.941156 1678 sys_catalog.cc:564] T 00000000000000000000000000000000 P 168bc8903141442bab766da1d878ea87 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 18:36:46.938858 1685 master_runner.cc:186] Error getting master registration for 127.19.194.62:35711: OK, instance_id { permanent_uuid: "168bc8903141442bab766da1d878ea87" instance_seqno: 1786041406441474 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260806 18:36:46.942207 1685 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
W20260806 18:36:46.946147 1899 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 168bc8903141442bab766da1d878ea87: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260806 18:36:46.946297 1899 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
W20260806 18:36:46.947642 1886 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 8fce36fcbb56469b9027d7bcfb145c91: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260806 18:36:46.947716 1886 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260806 18:36:46.949752 1606 master_runner.cc:263] No leader master found from master 168bc8903141442bab766da1d878ea87
I20260806 18:36:46.949821 1606 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
W20260806 18:36:47.101751 1793 tablet.cc:2369] T 00000000000000000000000000000000 P 72b869b5e7574895bb11643ba521af41: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260806 18:36:47.146199 1707 tablet.cc:2369] T 00000000000000000000000000000000 P 8fce36fcbb56469b9027d7bcfb145c91: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260806 18:36:47.188437 1630 tablet.cc:2369] T 00000000000000000000000000000000 P 168bc8903141442bab766da1d878ea87: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 18:36:47.242183 1857 raft_consensus.cc:482] T 00000000000000000000000000000000 P 72b869b5e7574895bb11643ba521af41 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 18:36:47.242398 1857 raft_consensus.cc:504] T 00000000000000000000000000000000 P 72b869b5e7574895bb11643ba521af41 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "168bc8903141442bab766da1d878ea87" member_type: VOTER last_known_addr { host: "127.19.194.62" port: 35711 } } peers { permanent_uuid: "8fce36fcbb56469b9027d7bcfb145c91" member_type: VOTER last_known_addr { host: "127.19.194.61" port: 37625 } } peers { permanent_uuid: "72b869b5e7574895bb11643ba521af41" member_type: VOTER last_known_addr { host: "127.19.194.60" port: 46189 } }
I20260806 18:36:47.243448 1857 leader_election.cc:288] T 00000000000000000000000000000000 P 72b869b5e7574895bb11643ba521af41 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 168bc8903141442bab766da1d878ea87 (127.19.194.62:35711), 8fce36fcbb56469b9027d7bcfb145c91 (127.19.194.61:37625)
I20260806 18:36:47.243904 1652 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "72b869b5e7574895bb11643ba521af41" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "168bc8903141442bab766da1d878ea87" is_pre_election: true
I20260806 18:36:47.243954 1729 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "72b869b5e7574895bb11643ba521af41" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8fce36fcbb56469b9027d7bcfb145c91" is_pre_election: true
I20260806 18:36:47.244208 1652 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 168bc8903141442bab766da1d878ea87 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 72b869b5e7574895bb11643ba521af41 in term 0.
I20260806 18:36:47.244230 1729 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 8fce36fcbb56469b9027d7bcfb145c91 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 72b869b5e7574895bb11643ba521af41 in term 0.
I20260806 18:36:47.244622 1789 leader_election.cc:302] T 00000000000000000000000000000000 P 72b869b5e7574895bb11643ba521af41 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 168bc8903141442bab766da1d878ea87, 72b869b5e7574895bb11643ba521af41; no voters:
I20260806 18:36:47.244894 1857 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 72b869b5e7574895bb11643ba521af41 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 18:36:47.244982 1857 raft_consensus.cc:482] T 00000000000000000000000000000000 P 72b869b5e7574895bb11643ba521af41 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 18:36:47.245014 1857 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 72b869b5e7574895bb11643ba521af41 [term 0 FOLLOWER]: Advancing to term 1
I20260806 18:36:47.246115 1857 raft_consensus.cc:504] T 00000000000000000000000000000000 P 72b869b5e7574895bb11643ba521af41 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "168bc8903141442bab766da1d878ea87" member_type: VOTER last_known_addr { host: "127.19.194.62" port: 35711 } } peers { permanent_uuid: "8fce36fcbb56469b9027d7bcfb145c91" member_type: VOTER last_known_addr { host: "127.19.194.61" port: 37625 } } peers { permanent_uuid: "72b869b5e7574895bb11643ba521af41" member_type: VOTER last_known_addr { host: "127.19.194.60" port: 46189 } }
I20260806 18:36:47.246448 1857 leader_election.cc:288] T 00000000000000000000000000000000 P 72b869b5e7574895bb11643ba521af41 [CANDIDATE]: Term 1 election: Requested vote from peers 168bc8903141442bab766da1d878ea87 (127.19.194.62:35711), 8fce36fcbb56469b9027d7bcfb145c91 (127.19.194.61:37625)
I20260806 18:36:47.246764 1652 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "72b869b5e7574895bb11643ba521af41" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "168bc8903141442bab766da1d878ea87"
I20260806 18:36:47.246793 1729 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "72b869b5e7574895bb11643ba521af41" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8fce36fcbb56469b9027d7bcfb145c91"
I20260806 18:36:47.246873 1652 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 168bc8903141442bab766da1d878ea87 [term 0 FOLLOWER]: Advancing to term 1
I20260806 18:36:47.246987 1729 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 8fce36fcbb56469b9027d7bcfb145c91 [term 0 FOLLOWER]: Advancing to term 1
I20260806 18:36:47.247932 1652 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 168bc8903141442bab766da1d878ea87 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 72b869b5e7574895bb11643ba521af41 in term 1.
I20260806 18:36:47.248073 1729 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 8fce36fcbb56469b9027d7bcfb145c91 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 72b869b5e7574895bb11643ba521af41 in term 1.
I20260806 18:36:47.248208 1789 leader_election.cc:302] T 00000000000000000000000000000000 P 72b869b5e7574895bb11643ba521af41 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 168bc8903141442bab766da1d878ea87, 72b869b5e7574895bb11643ba521af41; no voters:
I20260806 18:36:47.248378 1857 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 72b869b5e7574895bb11643ba521af41 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 18:36:47.248597 1857 raft_consensus.cc:686] T 00000000000000000000000000000000 P 72b869b5e7574895bb11643ba521af41 [term 1 LEADER]: Becoming Leader. State: Replica: 72b869b5e7574895bb11643ba521af41, State: Running, Role: LEADER
I20260806 18:36:47.248886 1857 consensus_queue.cc:237] T 00000000000000000000000000000000 P 72b869b5e7574895bb11643ba521af41 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "168bc8903141442bab766da1d878ea87" member_type: VOTER last_known_addr { host: "127.19.194.62" port: 35711 } } peers { permanent_uuid: "8fce36fcbb56469b9027d7bcfb145c91" member_type: VOTER last_known_addr { host: "127.19.194.61" port: 37625 } } peers { permanent_uuid: "72b869b5e7574895bb11643ba521af41" member_type: VOTER last_known_addr { host: "127.19.194.60" port: 46189 } }
I20260806 18:36:47.250823 1902 sys_catalog.cc:455] T 00000000000000000000000000000000 P 72b869b5e7574895bb11643ba521af41 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 72b869b5e7574895bb11643ba521af41. Latest consensus state: current_term: 1 leader_uuid: "72b869b5e7574895bb11643ba521af41" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "168bc8903141442bab766da1d878ea87" member_type: VOTER last_known_addr { host: "127.19.194.62" port: 35711 } } peers { permanent_uuid: "8fce36fcbb56469b9027d7bcfb145c91" member_type: VOTER last_known_addr { host: "127.19.194.61" port: 37625 } } peers { permanent_uuid: "72b869b5e7574895bb11643ba521af41" member_type: VOTER last_known_addr { host: "127.19.194.60" port: 46189 } } }
I20260806 18:36:47.250939 1902 sys_catalog.cc:458] T 00000000000000000000000000000000 P 72b869b5e7574895bb11643ba521af41 [sys.catalog]: This master's current role is: LEADER
I20260806 18:36:47.251348 1904 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 18:36:47.252183 1904 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 18:36:47.256412 1729 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 8fce36fcbb56469b9027d7bcfb145c91 [term 1 FOLLOWER]: Refusing update from remote peer 72b869b5e7574895bb11643ba521af41: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 18:36:47.256419 1652 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 168bc8903141442bab766da1d878ea87 [term 1 FOLLOWER]: Refusing update from remote peer 72b869b5e7574895bb11643ba521af41: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 18:36:47.256964 1902 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 72b869b5e7574895bb11643ba521af41 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8fce36fcbb56469b9027d7bcfb145c91" member_type: VOTER last_known_addr { host: "127.19.194.61" port: 37625 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 18:36:47.257138 1857 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 72b869b5e7574895bb11643ba521af41 [LEADER]: Connected to new peer: Peer: permanent_uuid: "168bc8903141442bab766da1d878ea87" member_type: VOTER last_known_addr { host: "127.19.194.62" port: 35711 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 18:36:47.260475 1874 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8fce36fcbb56469b9027d7bcfb145c91 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 72b869b5e7574895bb11643ba521af41. Latest consensus state: current_term: 1 leader_uuid: "72b869b5e7574895bb11643ba521af41" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "168bc8903141442bab766da1d878ea87" member_type: VOTER last_known_addr { host: "127.19.194.62" port: 35711 } } peers { permanent_uuid: "8fce36fcbb56469b9027d7bcfb145c91" member_type: VOTER last_known_addr { host: "127.19.194.61" port: 37625 } } peers { permanent_uuid: "72b869b5e7574895bb11643ba521af41" member_type: VOTER last_known_addr { host: "127.19.194.60" port: 46189 } } }
I20260806 18:36:47.260596 1874 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8fce36fcbb56469b9027d7bcfb145c91 [sys.catalog]: This master's current role is: FOLLOWER
I20260806 18:36:47.260566 1887 sys_catalog.cc:455] T 00000000000000000000000000000000 P 168bc8903141442bab766da1d878ea87 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 72b869b5e7574895bb11643ba521af41. Latest consensus state: current_term: 1 leader_uuid: "72b869b5e7574895bb11643ba521af41" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "168bc8903141442bab766da1d878ea87" member_type: VOTER last_known_addr { host: "127.19.194.62" port: 35711 } } peers { permanent_uuid: "8fce36fcbb56469b9027d7bcfb145c91" member_type: VOTER last_known_addr { host: "127.19.194.61" port: 37625 } } peers { permanent_uuid: "72b869b5e7574895bb11643ba521af41" member_type: VOTER last_known_addr { host: "127.19.194.60" port: 46189 } } }
I20260806 18:36:47.260717 1887 sys_catalog.cc:458] T 00000000000000000000000000000000 P 168bc8903141442bab766da1d878ea87 [sys.catalog]: This master's current role is: FOLLOWER
I20260806 18:36:47.263027 1857 sys_catalog.cc:455] T 00000000000000000000000000000000 P 72b869b5e7574895bb11643ba521af41 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "72b869b5e7574895bb11643ba521af41" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "168bc8903141442bab766da1d878ea87" member_type: VOTER last_known_addr { host: "127.19.194.62" port: 35711 } } peers { permanent_uuid: "8fce36fcbb56469b9027d7bcfb145c91" member_type: VOTER last_known_addr { host: "127.19.194.61" port: 37625 } } peers { permanent_uuid: "72b869b5e7574895bb11643ba521af41" member_type: VOTER last_known_addr { host: "127.19.194.60" port: 46189 } } }
I20260806 18:36:47.263227 1857 sys_catalog.cc:458] T 00000000000000000000000000000000 P 72b869b5e7574895bb11643ba521af41 [sys.catalog]: This master's current role is: LEADER
I20260806 18:36:47.263898 1887 sys_catalog.cc:455] T 00000000000000000000000000000000 P 168bc8903141442bab766da1d878ea87 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "72b869b5e7574895bb11643ba521af41" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "168bc8903141442bab766da1d878ea87" member_type: VOTER last_known_addr { host: "127.19.194.62" port: 35711 } } peers { permanent_uuid: "8fce36fcbb56469b9027d7bcfb145c91" member_type: VOTER last_known_addr { host: "127.19.194.61" port: 37625 } } peers { permanent_uuid: "72b869b5e7574895bb11643ba521af41" member_type: VOTER last_known_addr { host: "127.19.194.60" port: 46189 } } }
I20260806 18:36:47.263211 1902 sys_catalog.cc:455] T 00000000000000000000000000000000 P 72b869b5e7574895bb11643ba521af41 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "72b869b5e7574895bb11643ba521af41" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "168bc8903141442bab766da1d878ea87" member_type: VOTER last_known_addr { host: "127.19.194.62" port: 35711 } } peers { permanent_uuid: "8fce36fcbb56469b9027d7bcfb145c91" member_type: VOTER last_known_addr { host: "127.19.194.61" port: 37625 } } peers { permanent_uuid: "72b869b5e7574895bb11643ba521af41" member_type: VOTER last_known_addr { host: "127.19.194.60" port: 46189 } } }
I20260806 18:36:47.264000 1887 sys_catalog.cc:458] T 00000000000000000000000000000000 P 168bc8903141442bab766da1d878ea87 [sys.catalog]: This master's current role is: FOLLOWER
I20260806 18:36:47.264032 1902 sys_catalog.cc:458] T 00000000000000000000000000000000 P 72b869b5e7574895bb11643ba521af41 [sys.catalog]: This master's current role is: LEADER
I20260806 18:36:47.263998 1874 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8fce36fcbb56469b9027d7bcfb145c91 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "72b869b5e7574895bb11643ba521af41" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "168bc8903141442bab766da1d878ea87" member_type: VOTER last_known_addr { host: "127.19.194.62" port: 35711 } } peers { permanent_uuid: "8fce36fcbb56469b9027d7bcfb145c91" member_type: VOTER last_known_addr { host: "127.19.194.61" port: 37625 } } peers { permanent_uuid: "72b869b5e7574895bb11643ba521af41" member_type: VOTER last_known_addr { host: "127.19.194.60" port: 46189 } } }
I20260806 18:36:47.264103 1874 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8fce36fcbb56469b9027d7bcfb145c91 [sys.catalog]: This master's current role is: FOLLOWER
I20260806 18:36:47.264145 1907 mvcc.cc:205] Tried to move back new op lower bound from 7315625604116078592 to 7315625604094160896. Current Snapshot: MvccSnapshot[applied={T|T < 7315625604116078592}]
I20260806 18:36:47.263999 1906 mvcc.cc:205] Tried to move back new op lower bound from 7315625604116078592 to 7315625604094160896. Current Snapshot: MvccSnapshot[applied={T|T < 7315625604116078592}]
I20260806 18:36:47.265204 1904 catalog_manager.cc:1347] Generated new cluster ID: 35b067d34128413e8e1091fc926f61a8
I20260806 18:36:47.265313 1904 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 18:36:47.274533 1904 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 18:36:47.275624 1904 catalog_manager.cc:1504] Loading token signing keys...
I20260806 18:36:47.281980 1904 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 72b869b5e7574895bb11643ba521af41: Generated new TSK 0
I20260806 18:36:47.282660 1904 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 18:36:47.947590 1899 catalog_manager.cc:1259] Loaded cluster ID: 35b067d34128413e8e1091fc926f61a8
I20260806 18:36:47.947698 1899 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 168bc8903141442bab766da1d878ea87: loading cluster ID for follower catalog manager: success
I20260806 18:36:47.948910 1886 catalog_manager.cc:1259] Loaded cluster ID: 35b067d34128413e8e1091fc926f61a8
I20260806 18:36:47.948982 1886 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 8fce36fcbb56469b9027d7bcfb145c91: loading cluster ID for follower catalog manager: success
I20260806 18:36:47.949952 1899 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 168bc8903141442bab766da1d878ea87: acquiring CA information for follower catalog manager: success
I20260806 18:36:47.950805 1886 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 8fce36fcbb56469b9027d7bcfb145c91: acquiring CA information for follower catalog manager: success
I20260806 18:36:47.951305 1899 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 168bc8903141442bab766da1d878ea87: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260806 18:36:47.951822 1886 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 8fce36fcbb56469b9027d7bcfb145c91: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260806 18:36:48.339298 20232 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJW69tn/build/debug/bin/kudu with pid 1606
I20260806 18:36:48.347232 20232 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJW69tn/build/debug/bin/kudu with pid 1685
I20260806 18:36:48.353685 20232 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJW69tn/build/debug/bin/kudu with pid 1766
2026-08-06T18:36:48Z chronyd exiting
[ OK ] SecurityITest.IPKICACert (3267 ms)
[ RUN ] SecurityITest.IPKICACertWebServerEndPoints
Loading random data
Initializing database '/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786041246094603-20232-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 18:36:48 dist-test-slave-1xrh krb5kdc[1920](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 18:36:48 dist-test-slave-1xrh krb5kdc[1920](info): set up 2 sockets
Aug 06 18:36:48 dist-test-slave-1xrh krb5kdc[1920](info): commencing operation
krb5kdc: starting...
W20260806 18:36:49.401124 20232 mini_kdc.cc:121] Time spent starting KDC: real 1.025s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786041246094603-20232-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786041246094603-20232-0/krb5kdc/test-user.keytab.
Aug 06 18:36:49 dist-test-slave-1xrh krb5kdc[1920](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041409, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T18:36:49Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T18:36:49Z Disabled control of system clock
WARNING: no policy specified for kudu/127.19.194.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.19.194.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.19.194.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786041246094603-20232-0/krb5kdc/kudu_127.19.194.62.keytab.
Entry for principal kudu/127.19.194.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786041246094603-20232-0/krb5kdc/kudu_127.19.194.62.keytab.
I20260806 18:36:49.523998 20232 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJW69tn/build/debug/bin/kudu
/tmp/dist-test-taskJW69tn/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786041246094603-20232-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786041246094603-20232-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786041246094603-20232-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786041246094603-20232-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.19.194.62:43601
--webserver_interface=127.19.194.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786041246094603-20232-0/krb5kdc/kudu_127.19.194.62.keytab
--principal=kudu/127.19.194.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.19.194.20:41989
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.19.194.62:43601
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786041246094603-20232-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786041246094603-20232-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786041246094603-20232-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 18:36:49.630256 1934 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 18:36:49.630492 1934 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 18:36:49.630530 1934 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 18:36:49.634277 1934 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 18:36:49.634317 1934 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 18:36:49.634334 1934 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 18:36:49.634349 1934 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 18:36:49.634362 1934 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 18:36:49.639411 1934 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.194.20:41989
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786041246094603-20232-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786041246094603-20232-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.19.194.62:43601
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786041246094603-20232-0/krb5kdc/kudu_127.19.194.62.keytab
--principal=kudu/127.19.194.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.19.194.62:43601
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786041246094603-20232-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.19.194.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1934
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786041246094603-20232-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 17:59:13 UTC on ccycloud.quasar-wvsnzx.root.comops.site
build id 4571028
I20260806 18:36:49.640273 1934 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 18:36:49.641147 1934 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 18:36:49.649346 1939 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 18:36:49.649385 1934 server_base.cc:1034] running on GCE node
W20260806 18:36:49.649350 1942 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:36:49.649350 1940 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 18:36:49.650048 1934 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 18:36:49.651052 1934 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 18:36:49.652237 1934 hybrid_clock.cc:648] HybridClock initialized: now 1786041409652221 us; error 28 us; skew 500 ppm
Aug 06 18:36:49 dist-test-slave-1xrh krb5kdc[1920](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041409, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.194.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 18:36:49.654873 1934 init.cc:377] Logged in from keytab as kudu/127.19.194.62@KRBTEST.COM (short username kudu)
I20260806 18:36:49.655753 1934 webserver.cc:460] Webserver started at http://127.19.194.62:33415/ using document root <none> and password file <none>
I20260806 18:36:49.656287 1934 fs_manager.cc:359] Metadata directory not provided
I20260806 18:36:49.656378 1934 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 18:36:49.656607 1934 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 18:36:49.658557 1934 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786041246094603-20232-0/minicluster-data/master-0/data/instance:
uuid: "b6ffd971bda04625bc9dffd65f2f6747"
format_stamp: "Formatted at 2026-08-06 18:36:49 on dist-test-slave-1xrh"
server_key: "8f1d2c584d444429f1dd4bf0d221bd20"
server_key_iv: "dc905a60b312f8f92cb5d9554feb9362"
server_key_version: "encryptionkey@0"
I20260806 18:36:49.659192 1934 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786041246094603-20232-0/minicluster-data/master-0/wal/instance:
uuid: "b6ffd971bda04625bc9dffd65f2f6747"
format_stamp: "Formatted at 2026-08-06 18:36:49 on dist-test-slave-1xrh"
server_key: "8f1d2c584d444429f1dd4bf0d221bd20"
server_key_iv: "dc905a60b312f8f92cb5d9554feb9362"
server_key_version: "encryptionkey@0"
I20260806 18:36:49.662751 1934 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.002s sys 0.000s
I20260806 18:36:49.665257 1949 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 18:36:49.666422 1934 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260806 18:36:49.666576 1934 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786041246094603-20232-0/minicluster-data/master-0/data,/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786041246094603-20232-0/minicluster-data/master-0/wal
uuid: "b6ffd971bda04625bc9dffd65f2f6747"
format_stamp: "Formatted at 2026-08-06 18:36:49 on dist-test-slave-1xrh"
server_key: "8f1d2c584d444429f1dd4bf0d221bd20"
server_key_iv: "dc905a60b312f8f92cb5d9554feb9362"
server_key_version: "encryptionkey@0"
I20260806 18:36:49.666697 1934 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786041246094603-20232-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786041246094603-20232-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786041246094603-20232-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 18:36:49.698521 1934 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 18:36:49.701994 1934 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 18:36:49.702214 1934 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 18:36:49.709466 1934 rpc_server.cc:307] RPC server started. Bound to: 127.19.194.62:43601
I20260806 18:36:49.709527 2005 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.194.62:43601 every 8 connection(s)
I20260806 18:36:49.710439 1934 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786041246094603-20232-0/minicluster-data/master-0/data/info.pb
I20260806 18:36:49.713164 2006 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 18:36:49.718470 2006 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b6ffd971bda04625bc9dffd65f2f6747: Bootstrap starting.
I20260806 18:36:49.720073 20232 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJW69tn/build/debug/bin/kudu as pid 1934
I20260806 18:36:49.720192 20232 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786041246094603-20232-0/minicluster-data/master-0/wal/instance
I20260806 18:36:49.720465 20232 external_mini_cluster.cc:1428] Setting key a5370672676e6e03dbf761daf80b970a
I20260806 18:36:49.721118 2006 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P b6ffd971bda04625bc9dffd65f2f6747: Neither blocks nor log segments found. Creating new log.
I20260806 18:36:49.721940 2006 log.cc:824] T 00000000000000000000000000000000 P b6ffd971bda04625bc9dffd65f2f6747: Log is configured to *not* fsync() on all Append() calls
I20260806 18:36:49.724112 2006 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b6ffd971bda04625bc9dffd65f2f6747: No bootstrap required, opened a new log
I20260806 18:36:49.727440 2006 raft_consensus.cc:348] T 00000000000000000000000000000000 P b6ffd971bda04625bc9dffd65f2f6747 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b6ffd971bda04625bc9dffd65f2f6747" member_type: VOTER last_known_addr { host: "127.19.194.62" port: 43601 } }
I20260806 18:36:49.727665 2006 raft_consensus.cc:374] T 00000000000000000000000000000000 P b6ffd971bda04625bc9dffd65f2f6747 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 18:36:49.727746 2006 raft_consensus.cc:729] T 00000000000000000000000000000000 P b6ffd971bda04625bc9dffd65f2f6747 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b6ffd971bda04625bc9dffd65f2f6747, State: Initialized, Role: FOLLOWER
I20260806 18:36:49.728214 2006 consensus_queue.cc:260] T 00000000000000000000000000000000 P b6ffd971bda04625bc9dffd65f2f6747 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b6ffd971bda04625bc9dffd65f2f6747" member_type: VOTER last_known_addr { host: "127.19.194.62" port: 43601 } }
I20260806 18:36:49.728355 2006 raft_consensus.cc:388] T 00000000000000000000000000000000 P b6ffd971bda04625bc9dffd65f2f6747 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 18:36:49.728441 2006 raft_consensus.cc:482] T 00000000000000000000000000000000 P b6ffd971bda04625bc9dffd65f2f6747 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 18:36:49.728534 2006 raft_consensus.cc:3037] T 00000000000000000000000000000000 P b6ffd971bda04625bc9dffd65f2f6747 [term 0 FOLLOWER]: Advancing to term 1
Aug 06 18:36:49 dist-test-slave-1xrh krb5kdc[1920](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041409, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.19.194.62@KRBTEST.COM
I20260806 18:36:49.729457 2006 raft_consensus.cc:504] T 00000000000000000000000000000000 P b6ffd971bda04625bc9dffd65f2f6747 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b6ffd971bda04625bc9dffd65f2f6747" member_type: VOTER last_known_addr { host: "127.19.194.62" port: 43601 } }
I20260806 18:36:49.729780 2006 leader_election.cc:302] T 00000000000000000000000000000000 P b6ffd971bda04625bc9dffd65f2f6747 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: b6ffd971bda04625bc9dffd65f2f6747; no voters:
I20260806 18:36:49.730144 2006 leader_election.cc:288] T 00000000000000000000000000000000 P b6ffd971bda04625bc9dffd65f2f6747 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 18:36:49.730191 2011 raft_consensus.cc:2781] T 00000000000000000000000000000000 P b6ffd971bda04625bc9dffd65f2f6747 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 18:36:49.730408 2011 raft_consensus.cc:686] T 00000000000000000000000000000000 P b6ffd971bda04625bc9dffd65f2f6747 [term 1 LEADER]: Becoming Leader. State: Replica: b6ffd971bda04625bc9dffd65f2f6747, State: Running, Role: LEADER
I20260806 18:36:49.730690 2011 consensus_queue.cc:237] T 00000000000000000000000000000000 P b6ffd971bda04625bc9dffd65f2f6747 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b6ffd971bda04625bc9dffd65f2f6747" member_type: VOTER last_known_addr { host: "127.19.194.62" port: 43601 } }
I20260806 18:36:49.731335 2006 sys_catalog.cc:564] T 00000000000000000000000000000000 P b6ffd971bda04625bc9dffd65f2f6747 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 18:36:49.731775 2012 sys_catalog.cc:455] T 00000000000000000000000000000000 P b6ffd971bda04625bc9dffd65f2f6747 [sys.catalog]: SysCatalogTable state changed. Reason: New leader b6ffd971bda04625bc9dffd65f2f6747. Latest consensus state: current_term: 1 leader_uuid: "b6ffd971bda04625bc9dffd65f2f6747" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b6ffd971bda04625bc9dffd65f2f6747" member_type: VOTER last_known_addr { host: "127.19.194.62" port: 43601 } } }
I20260806 18:36:49.731894 2012 sys_catalog.cc:458] T 00000000000000000000000000000000 P b6ffd971bda04625bc9dffd65f2f6747 [sys.catalog]: This master's current role is: LEADER
I20260806 18:36:49.732115 2013 sys_catalog.cc:455] T 00000000000000000000000000000000 P b6ffd971bda04625bc9dffd65f2f6747 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "b6ffd971bda04625bc9dffd65f2f6747" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b6ffd971bda04625bc9dffd65f2f6747" member_type: VOTER last_known_addr { host: "127.19.194.62" port: 43601 } } }
I20260806 18:36:49.732226 2013 sys_catalog.cc:458] T 00000000000000000000000000000000 P b6ffd971bda04625bc9dffd65f2f6747 [sys.catalog]: This master's current role is: LEADER
I20260806 18:36:49.732707 2016 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 18:36:49.735234 2009 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:36:49.721507 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60524 (local address 127.19.194.62:43601)
0806 18:36:49.721987 (+ 480us) server_negotiation.cc:206] Beginning negotiation
0806 18:36:49.722007 (+ 20us) server_negotiation.cc:399] Waiting for connection header
0806 18:36:49.722050 (+ 43us) server_negotiation.cc:407] Connection header received
0806 18:36:49.722808 (+ 758us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:36:49.722830 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:36:49.723220 (+ 390us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:36:49.723582 (+ 362us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:36:49.724332 (+ 750us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:36:49.725294 (+ 962us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:36:49.726162 (+ 868us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:36:49.726635 (+ 473us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:36:49.726809 (+ 174us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:36:49.729534 (+ 2725us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:36:49.729545 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:36:49.729556 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 18:36:49.729603 (+ 47us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:36:49.731723 (+ 2120us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:36:49.732254 (+ 531us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:36:49.732259 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:36:49.732267 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:36:49.732403 (+ 136us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:36:49.732762 (+ 359us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:36:49.732767 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:36:49.732771 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:36:49.733244 (+ 473us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:36:49.733481 (+ 237us) server_negotiation.cc:1035] Waiting for connection context
0806 18:36:49.733962 (+ 481us) server_negotiation.cc:299] Negotiation successful
0806 18:36:49.734279 (+ 317us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":327,"thread_start_us":138,"threads_started":1}
I20260806 18:36:49.736354 2016 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 18:36:49.739432 20232 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260806 18:36:49.744349 2016 catalog_manager.cc:1347] Generated new cluster ID: c5254b9ef3574c7da685f00eb64c9775
I20260806 18:36:49.744412 2016 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 18:36:49.767002 2016 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 18:36:49.767936 2016 catalog_manager.cc:1504] Loading token signing keys...
I20260806 18:36:49.780545 2016 catalog_manager.cc:5868] T 00000000000000000000000000000000 P b6ffd971bda04625bc9dffd65f2f6747: Generated new TSK 0
I20260806 18:36:49.781234 2016 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 18:36:49.795347 20232 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJW69tn/build/debug/bin/kudu with pid 1934
2026-08-06T18:36:49Z chronyd exiting
[ OK ] SecurityITest.IPKICACertWebServerEndPoints (1440 ms)
[ RUN ] SecurityITest.TestEnableAndDisableEncryptedIPKICert
Loading random data
Initializing database '/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786041246094603-20232-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 18:36:49 dist-test-slave-1xrh krb5kdc[2033](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 18:36:49 dist-test-slave-1xrh krb5kdc[2033](info): set up 2 sockets
Aug 06 18:36:49 dist-test-slave-1xrh krb5kdc[2033](info): commencing operation
krb5kdc: starting...
W20260806 18:36:50.852483 20232 mini_kdc.cc:121] Time spent starting KDC: real 1.035s user 0.003s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786041246094603-20232-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786041246094603-20232-0/krb5kdc/test-user.keytab.
Aug 06 18:36:50 dist-test-slave-1xrh krb5kdc[2033](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041410, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T18:36:50Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T18:36:50Z Disabled control of system clock
WARNING: no policy specified for kudu/127.19.194.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.19.194.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.19.194.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786041246094603-20232-0/krb5kdc/kudu_127.19.194.62.keytab.
Entry for principal kudu/127.19.194.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786041246094603-20232-0/krb5kdc/kudu_127.19.194.62.keytab.
I20260806 18:36:50.973973 20232 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJW69tn/build/debug/bin/kudu
/tmp/dist-test-taskJW69tn/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786041246094603-20232-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786041246094603-20232-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786041246094603-20232-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786041246094603-20232-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.19.194.62:40989
--webserver_interface=127.19.194.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786041246094603-20232-0/krb5kdc/kudu_127.19.194.62.keytab
--principal=kudu/127.19.194.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.19.194.20:38847
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.19.194.62:40989
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786041246094603-20232-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786041246094603-20232-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786041246094603-20232-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 18:36:51.080817 2047 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 18:36:51.081070 2047 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 18:36:51.081110 2047 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 18:36:51.084942 2047 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 18:36:51.084988 2047 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 18:36:51.085008 2047 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 18:36:51.085029 2047 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 18:36:51.085045 2047 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 18:36:51.089926 2047 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.194.20:38847
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786041246094603-20232-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786041246094603-20232-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.19.194.62:40989
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786041246094603-20232-0/krb5kdc/kudu_127.19.194.62.keytab
--principal=kudu/127.19.194.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.19.194.62:40989
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786041246094603-20232-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.19.194.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2047
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786041246094603-20232-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 17:59:13 UTC on ccycloud.quasar-wvsnzx.root.comops.site
build id 4571028
I20260806 18:36:51.090852 2047 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 18:36:51.091797 2047 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 18:36:51.099998 2052 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:36:51.100018 2055 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:36:51.100006 2053 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 18:36:51.100258 2047 server_base.cc:1034] running on GCE node
I20260806 18:36:51.100780 2047 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 18:36:51.101812 2047 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 18:36:51.102972 2047 hybrid_clock.cc:648] HybridClock initialized: now 1786041411102954 us; error 29 us; skew 500 ppm
Aug 06 18:36:51 dist-test-slave-1xrh krb5kdc[2033](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041411, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.194.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 18:36:51.112843 2047 init.cc:377] Logged in from keytab as kudu/127.19.194.62@KRBTEST.COM (short username kudu)
I20260806 18:36:51.113780 2047 webserver.cc:460] Webserver started at http://127.19.194.62:33889/ using document root <none> and password file <none>
I20260806 18:36:51.114329 2047 fs_manager.cc:359] Metadata directory not provided
I20260806 18:36:51.114390 2047 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 18:36:51.114647 2047 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 18:36:51.116740 2047 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786041246094603-20232-0/minicluster-data/master-0/data/instance:
uuid: "422f52e3e25c47249416f4d5b21c77be"
format_stamp: "Formatted at 2026-08-06 18:36:51 on dist-test-slave-1xrh"
server_key: "5a4c117b1fc3712dd62d3e9f64694880"
server_key_iv: "78c55a7345d150ac7b0dbed4e7f1b0c6"
server_key_version: "encryptionkey@0"
I20260806 18:36:51.117341 2047 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786041246094603-20232-0/minicluster-data/master-0/wal/instance:
uuid: "422f52e3e25c47249416f4d5b21c77be"
format_stamp: "Formatted at 2026-08-06 18:36:51 on dist-test-slave-1xrh"
server_key: "5a4c117b1fc3712dd62d3e9f64694880"
server_key_iv: "78c55a7345d150ac7b0dbed4e7f1b0c6"
server_key_version: "encryptionkey@0"
I20260806 18:36:51.121050 2047 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.005s sys 0.000s
I20260806 18:36:51.123351 2063 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 18:36:51.124607 2047 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260806 18:36:51.124753 2047 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786041246094603-20232-0/minicluster-data/master-0/data,/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786041246094603-20232-0/minicluster-data/master-0/wal
uuid: "422f52e3e25c47249416f4d5b21c77be"
format_stamp: "Formatted at 2026-08-06 18:36:51 on dist-test-slave-1xrh"
server_key: "5a4c117b1fc3712dd62d3e9f64694880"
server_key_iv: "78c55a7345d150ac7b0dbed4e7f1b0c6"
server_key_version: "encryptionkey@0"
I20260806 18:36:51.124864 2047 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786041246094603-20232-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786041246094603-20232-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786041246094603-20232-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 18:36:51.138464 2047 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 18:36:51.141687 2047 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 18:36:51.141880 2047 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 18:36:51.149061 2047 rpc_server.cc:307] RPC server started. Bound to: 127.19.194.62:40989
I20260806 18:36:51.149360 2117 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.194.62:40989 every 8 connection(s)
I20260806 18:36:51.149988 2047 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786041246094603-20232-0/minicluster-data/master-0/data/info.pb
I20260806 18:36:51.152518 2118 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 18:36:51.157693 2118 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 422f52e3e25c47249416f4d5b21c77be: Bootstrap starting.
I20260806 18:36:51.159916 20232 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJW69tn/build/debug/bin/kudu as pid 2047
I20260806 18:36:51.160070 20232 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786041246094603-20232-0/minicluster-data/master-0/wal/instance
I20260806 18:36:51.160236 2118 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 422f52e3e25c47249416f4d5b21c77be: Neither blocks nor log segments found. Creating new log.
I20260806 18:36:51.160400 20232 external_mini_cluster.cc:1428] Setting key 70663b5135e95b07fc0714b54e4362aa
I20260806 18:36:51.161069 2118 log.cc:824] T 00000000000000000000000000000000 P 422f52e3e25c47249416f4d5b21c77be: Log is configured to *not* fsync() on all Append() calls
I20260806 18:36:51.162921 2118 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 422f52e3e25c47249416f4d5b21c77be: No bootstrap required, opened a new log
I20260806 18:36:51.167086 2118 raft_consensus.cc:348] T 00000000000000000000000000000000 P 422f52e3e25c47249416f4d5b21c77be [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "422f52e3e25c47249416f4d5b21c77be" member_type: VOTER last_known_addr { host: "127.19.194.62" port: 40989 } }
I20260806 18:36:51.167397 2118 raft_consensus.cc:374] T 00000000000000000000000000000000 P 422f52e3e25c47249416f4d5b21c77be [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 18:36:51.167464 2118 raft_consensus.cc:729] T 00000000000000000000000000000000 P 422f52e3e25c47249416f4d5b21c77be [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 422f52e3e25c47249416f4d5b21c77be, State: Initialized, Role: FOLLOWER
I20260806 18:36:51.167973 2118 consensus_queue.cc:260] T 00000000000000000000000000000000 P 422f52e3e25c47249416f4d5b21c77be [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "422f52e3e25c47249416f4d5b21c77be" member_type: VOTER last_known_addr { host: "127.19.194.62" port: 40989 } }
I20260806 18:36:51.168179 2118 raft_consensus.cc:388] T 00000000000000000000000000000000 P 422f52e3e25c47249416f4d5b21c77be [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 18:36:51.168263 2118 raft_consensus.cc:482] T 00000000000000000000000000000000 P 422f52e3e25c47249416f4d5b21c77be [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 18:36:51.168365 2118 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 422f52e3e25c47249416f4d5b21c77be [term 0 FOLLOWER]: Advancing to term 1
Aug 06 18:36:51 dist-test-slave-1xrh krb5kdc[2033](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041410, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.19.194.62@KRBTEST.COM
I20260806 18:36:51.169265 2118 raft_consensus.cc:504] T 00000000000000000000000000000000 P 422f52e3e25c47249416f4d5b21c77be [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "422f52e3e25c47249416f4d5b21c77be" member_type: VOTER last_known_addr { host: "127.19.194.62" port: 40989 } }
I20260806 18:36:51.169627 2118 leader_election.cc:302] T 00000000000000000000000000000000 P 422f52e3e25c47249416f4d5b21c77be [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 422f52e3e25c47249416f4d5b21c77be; no voters:
I20260806 18:36:51.169972 2118 leader_election.cc:288] T 00000000000000000000000000000000 P 422f52e3e25c47249416f4d5b21c77be [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 18:36:51.170303 2123 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 422f52e3e25c47249416f4d5b21c77be [term 1 FOLLOWER]: Leader election won for term 1
I20260806 18:36:51.170512 2123 raft_consensus.cc:686] T 00000000000000000000000000000000 P 422f52e3e25c47249416f4d5b21c77be [term 1 LEADER]: Becoming Leader. State: Replica: 422f52e3e25c47249416f4d5b21c77be, State: Running, Role: LEADER
I20260806 18:36:51.171108 2123 consensus_queue.cc:237] T 00000000000000000000000000000000 P 422f52e3e25c47249416f4d5b21c77be [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "422f52e3e25c47249416f4d5b21c77be" member_type: VOTER last_known_addr { host: "127.19.194.62" port: 40989 } }
I20260806 18:36:51.171334 2118 sys_catalog.cc:564] T 00000000000000000000000000000000 P 422f52e3e25c47249416f4d5b21c77be [sys.catalog]: configured and running, proceeding with master startup.
I20260806 18:36:51.172976 2124 sys_catalog.cc:455] T 00000000000000000000000000000000 P 422f52e3e25c47249416f4d5b21c77be [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "422f52e3e25c47249416f4d5b21c77be" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "422f52e3e25c47249416f4d5b21c77be" member_type: VOTER last_known_addr { host: "127.19.194.62" port: 40989 } } }
I20260806 18:36:51.173110 2124 sys_catalog.cc:458] T 00000000000000000000000000000000 P 422f52e3e25c47249416f4d5b21c77be [sys.catalog]: This master's current role is: LEADER
I20260806 18:36:51.173364 2125 sys_catalog.cc:455] T 00000000000000000000000000000000 P 422f52e3e25c47249416f4d5b21c77be [sys.catalog]: SysCatalogTable state changed. Reason: New leader 422f52e3e25c47249416f4d5b21c77be. Latest consensus state: current_term: 1 leader_uuid: "422f52e3e25c47249416f4d5b21c77be" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "422f52e3e25c47249416f4d5b21c77be" member_type: VOTER last_known_addr { host: "127.19.194.62" port: 40989 } } }
I20260806 18:36:51.173466 2125 sys_catalog.cc:458] T 00000000000000000000000000000000 P 422f52e3e25c47249416f4d5b21c77be [sys.catalog]: This master's current role is: LEADER
I20260806 18:36:51.173761 2137 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 18:36:51.175019 2122 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:36:51.161475 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59898 (local address 127.19.194.62:40989)
0806 18:36:51.161831 (+ 356us) server_negotiation.cc:206] Beginning negotiation
0806 18:36:51.161851 (+ 20us) server_negotiation.cc:399] Waiting for connection header
0806 18:36:51.161898 (+ 47us) server_negotiation.cc:407] Connection header received
0806 18:36:51.162492 (+ 594us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:36:51.162505 (+ 13us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:36:51.162783 (+ 278us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:36:51.163083 (+ 300us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:36:51.163982 (+ 899us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:36:51.164922 (+ 940us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:36:51.166205 (+ 1283us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:36:51.166652 (+ 447us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:36:51.166766 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:36:51.169617 (+ 2851us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:36:51.169623 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:36:51.169639 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 18:36:51.169675 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:36:51.171800 (+ 2125us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:36:51.172337 (+ 537us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:36:51.172343 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:36:51.172349 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:36:51.172496 (+ 147us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:36:51.172841 (+ 345us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:36:51.172844 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:36:51.172846 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:36:51.173368 (+ 522us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:36:51.173599 (+ 231us) server_negotiation.cc:1035] Waiting for connection context
0806 18:36:51.174001 (+ 402us) server_negotiation.cc:299] Negotiation successful
0806 18:36:51.174313 (+ 312us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":223,"thread_start_us":103,"threads_started":1}
I20260806 18:36:51.177155 2137 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 18:36:51.182390 2137 catalog_manager.cc:1347] Generated new cluster ID: 4eef91136e8e4ae8b146e3827730dab6
I20260806 18:36:51.182454 2137 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 18:36:51.202384 2137 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 18:36:51.203321 2137 catalog_manager.cc:1504] Loading token signing keys...
I20260806 18:36:51.210603 2137 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 422f52e3e25c47249416f4d5b21c77be: Generated new TSK 0
I20260806 18:36:51.211234 2137 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.19.194.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.19.194.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.19.194.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786041246094603-20232-0/krb5kdc/kudu_127.19.194.1.keytab.
Entry for principal kudu/127.19.194.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786041246094603-20232-0/krb5kdc/kudu_127.19.194.1.keytab.
I20260806 18:36:51.243952 20232 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJW69tn/build/debug/bin/kudu
/tmp/dist-test-taskJW69tn/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786041246094603-20232-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786041246094603-20232-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786041246094603-20232-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786041246094603-20232-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.19.194.1:0
--local_ip_for_outbound_sockets=127.19.194.1
--webserver_interface=127.19.194.1
--webserver_port=0
--tserver_master_addrs=127.19.194.62:40989
--keytab_file=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786041246094603-20232-0/krb5kdc/kudu_127.19.194.1.keytab
--principal=kudu/127.19.194.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.19.194.20:38847
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786041246094603-20232-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786041246094603-20232-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786041246094603-20232-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 18:36:51.356514 2144 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 18:36:51.356755 2144 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 18:36:51.356801 2144 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 18:36:51.360836 2144 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 18:36:51.360885 2144 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 18:36:51.360984 2144 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.19.194.1
I20260806 18:36:51.365999 2144 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.194.20:38847
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786041246094603-20232-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786041246094603-20232-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786041246094603-20232-0/krb5kdc/kudu_127.19.194.1.keytab
--principal=kudu/127.19.194.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.19.194.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786041246094603-20232-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.19.194.1
--webserver_port=0
--tserver_master_addrs=127.19.194.62:40989
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2144
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.19.194.1
--log_dir=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786041246094603-20232-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 17:59:13 UTC on ccycloud.quasar-wvsnzx.root.comops.site
build id 4571028
I20260806 18:36:51.366954 2144 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 18:36:51.367993 2144 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 18:36:51.377684 2149 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 18:36:51.377718 2144 server_base.cc:1034] running on GCE node
W20260806 18:36:51.377645 2152 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:36:51.377669 2150 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 18:36:51.378276 2144 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 18:36:51.378906 2144 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 18:36:51.380096 2144 hybrid_clock.cc:648] HybridClock initialized: now 1786041411380106 us; error 52 us; skew 500 ppm
Aug 06 18:36:51 dist-test-slave-1xrh krb5kdc[2033](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041411, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.194.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 18:36:51.382856 2144 init.cc:377] Logged in from keytab as kudu/127.19.194.1@KRBTEST.COM (short username kudu)
I20260806 18:36:51.383679 2144 webserver.cc:460] Webserver started at http://127.19.194.1:38357/ using document root <none> and password file <none>
I20260806 18:36:51.384181 2144 fs_manager.cc:359] Metadata directory not provided
I20260806 18:36:51.384238 2144 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 18:36:51.384399 2144 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 18:36:51.386307 2144 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786041246094603-20232-0/minicluster-data/ts-0/data/instance:
uuid: "e5ac26f8baab47dd8096af286ff02861"
format_stamp: "Formatted at 2026-08-06 18:36:51 on dist-test-slave-1xrh"
server_key: "ecfbc98d4b6d20352ff87523e6e75ace"
server_key_iv: "4d26e6cb9a1633f9b7144c6be6bdbdc2"
server_key_version: "encryptionkey@0"
I20260806 18:36:51.386816 2144 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786041246094603-20232-0/minicluster-data/ts-0/wal/instance:
uuid: "e5ac26f8baab47dd8096af286ff02861"
format_stamp: "Formatted at 2026-08-06 18:36:51 on dist-test-slave-1xrh"
server_key: "ecfbc98d4b6d20352ff87523e6e75ace"
server_key_iv: "4d26e6cb9a1633f9b7144c6be6bdbdc2"
server_key_version: "encryptionkey@0"
I20260806 18:36:51.390268 2144 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.006s sys 0.000s
I20260806 18:36:51.392550 2159 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 18:36:51.393901 2144 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.000s sys 0.001s
I20260806 18:36:51.394016 2144 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786041246094603-20232-0/minicluster-data/ts-0/data,/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786041246094603-20232-0/minicluster-data/ts-0/wal
uuid: "e5ac26f8baab47dd8096af286ff02861"
format_stamp: "Formatted at 2026-08-06 18:36:51 on dist-test-slave-1xrh"
server_key: "ecfbc98d4b6d20352ff87523e6e75ace"
server_key_iv: "4d26e6cb9a1633f9b7144c6be6bdbdc2"
server_key_version: "encryptionkey@0"
I20260806 18:36:51.394116 2144 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786041246094603-20232-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786041246094603-20232-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786041246094603-20232-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 18:36:51.416514 2144 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 18:36:51.419864 2144 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 18:36:51.420073 2144 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 18:36:51.420687 2144 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 18:36:51.421574 2144 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 18:36:51.421633 2144 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 18:36:51.421682 2144 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 18:36:51.421710 2144 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 18:36:51.431023 2144 rpc_server.cc:307] RPC server started. Bound to: 127.19.194.1:42955
I20260806 18:36:51.431114 2277 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.194.1:42955 every 8 connection(s)
I20260806 18:36:51.431973 2144 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786041246094603-20232-0/minicluster-data/ts-0/data/info.pb
I20260806 18:36:51.439920 20232 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJW69tn/build/debug/bin/kudu as pid 2144
I20260806 18:36:51.440061 20232 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786041246094603-20232-0/minicluster-data/ts-0/wal/instance
I20260806 18:36:51.440443 20232 external_mini_cluster.cc:1428] Setting key c6d1e3a761470a1f05d25f09cccd70e4
Aug 06 18:36:51 dist-test-slave-1xrh krb5kdc[2033](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041411, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.194.1@KRBTEST.COM for kudu/127.19.194.62@KRBTEST.COM
I20260806 18:36:51.446076 2122 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:36:51.433752 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.194.1:45951 (local address 127.19.194.62:40989)
0806 18:36:51.433940 (+ 188us) server_negotiation.cc:206] Beginning negotiation
0806 18:36:51.433946 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 18:36:51.434860 (+ 914us) server_negotiation.cc:407] Connection header received
0806 18:36:51.436277 (+ 1417us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:36:51.436280 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:36:51.436342 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:36:51.436421 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:36:51.437529 (+ 1108us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:36:51.438053 (+ 524us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:36:51.439150 (+ 1097us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:36:51.439381 (+ 231us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:36:51.439494 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:36:51.442321 (+ 2827us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:36:51.442324 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:36:51.442331 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 18:36:51.442353 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:36:51.444027 (+ 1674us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:36:51.444521 (+ 494us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:36:51.444527 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:36:51.444531 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:36:51.444629 (+ 98us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:36:51.444958 (+ 329us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:36:51.444962 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:36:51.444963 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:36:51.445185 (+ 222us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:36:51.445268 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0806 18:36:51.445866 (+ 598us) server_negotiation.cc:299] Negotiation successful
0806 18:36:51.445960 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260806 18:36:51.447327 2280 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:36:51.434254 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.194.62:40989 (local address 127.19.194.1:45951)
0806 18:36:51.434692 (+ 438us) negotiation.cc:107] Waiting for socket to connect
0806 18:36:51.434734 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0806 18:36:51.436053 (+ 1319us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:36:51.436576 (+ 523us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:36:51.436583 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:36:51.436912 (+ 329us) client_negotiation.cc:189] Negotiated authn=SASL
0806 18:36:51.437385 (+ 473us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:36:51.437398 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:36:51.438182 (+ 784us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:36:51.438185 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:36:51.439000 (+ 815us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:36:51.439009 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:36:51.439530 (+ 521us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:36:51.439534 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:36:51.439737 (+ 203us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:36:51.440351 (+ 614us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 18:36:51.440382 (+ 31us) client_negotiation.cc:656] calling sasl_client_start()
0806 18:36:51.442163 (+ 1781us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 18:36:51.444162 (+ 1999us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:36:51.444166 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:36:51.444177 (+ 11us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:36:51.444422 (+ 245us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:36:51.444732 (+ 310us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:36:51.444734 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:36:51.444735 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:36:51.444856 (+ 121us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:36:51.445300 (+ 444us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 18:36:51.445309 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 18:36:51.445608 (+ 299us) client_negotiation.cc:769] Sending connection context
0806 18:36:51.445815 (+ 207us) client_negotiation.cc:240] Negotiation successful
0806 18:36:51.446165 (+ 350us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":257,"thread_start_us":117,"threads_started":1}
I20260806 18:36:51.448534 2278 heartbeater.cc:347] Connected to a master server at 127.19.194.62:40989
I20260806 18:36:51.448764 2278 heartbeater.cc:464] Registering TS with master...
I20260806 18:36:51.449205 2278 heartbeater.cc:511] Master 127.19.194.62:40989 requested a full tablet report, sending...
I20260806 18:36:51.450639 2080 ts_manager.cc:194] Registered new tserver with Master: e5ac26f8baab47dd8096af286ff02861 (127.19.194.1:42955)
WARNING: no policy specified for kudu/127.19.194.2@KRBTEST.COM; defaulting to no policy
I20260806 18:36:51.451881 2080 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.19.194.1@KRBTEST.COM'} at 127.19.194.1:45951
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.19.194.2@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.19.194.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786041246094603-20232-0/krb5kdc/kudu_127.19.194.2.keytab.
Entry for principal kudu/127.19.194.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786041246094603-20232-0/krb5kdc/kudu_127.19.194.2.keytab.
I20260806 18:36:51.470667 20232 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJW69tn/build/debug/bin/kudu
/tmp/dist-test-taskJW69tn/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786041246094603-20232-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786041246094603-20232-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786041246094603-20232-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786041246094603-20232-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.19.194.2:0
--local_ip_for_outbound_sockets=127.19.194.2
--webserver_interface=127.19.194.2
--webserver_port=0
--tserver_master_addrs=127.19.194.62:40989
--keytab_file=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786041246094603-20232-0/krb5kdc/kudu_127.19.194.2.keytab
--principal=kudu/127.19.194.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.19.194.20:38847
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786041246094603-20232-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786041246094603-20232-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786041246094603-20232-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 18:36:51.579423 2283 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 18:36:51.579648 2283 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 18:36:51.579689 2283 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 18:36:51.583501 2283 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 18:36:51.583545 2283 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 18:36:51.583599 2283 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.19.194.2
I20260806 18:36:51.588550 2283 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.194.20:38847
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786041246094603-20232-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786041246094603-20232-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786041246094603-20232-0/krb5kdc/kudu_127.19.194.2.keytab
--principal=kudu/127.19.194.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.19.194.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786041246094603-20232-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.19.194.2
--webserver_port=0
--tserver_master_addrs=127.19.194.62:40989
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2283
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.19.194.2
--log_dir=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786041246094603-20232-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 17:59:13 UTC on ccycloud.quasar-wvsnzx.root.comops.site
build id 4571028
I20260806 18:36:51.589448 2283 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 18:36:51.590340 2283 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 18:36:51.599691 2289 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 18:36:51.599992 2283 server_base.cc:1034] running on GCE node
W20260806 18:36:51.600077 2288 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:36:51.600337 2291 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 18:36:51.601271 2283 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 18:36:51.602082 2283 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 18:36:51.603252 2283 hybrid_clock.cc:648] HybridClock initialized: now 1786041411603232 us; error 34 us; skew 500 ppm
Aug 06 18:36:51 dist-test-slave-1xrh krb5kdc[2033](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041411, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.194.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 18:36:51.607285 2283 init.cc:377] Logged in from keytab as kudu/127.19.194.2@KRBTEST.COM (short username kudu)
I20260806 18:36:51.608367 2283 webserver.cc:460] Webserver started at http://127.19.194.2:42471/ using document root <none> and password file <none>
I20260806 18:36:51.609058 2283 fs_manager.cc:359] Metadata directory not provided
I20260806 18:36:51.609153 2283 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 18:36:51.609390 2283 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 18:36:51.612334 2283 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786041246094603-20232-0/minicluster-data/ts-1/data/instance:
uuid: "61d7bb0f87d04b1b85aa1dbf30e6bc74"
format_stamp: "Formatted at 2026-08-06 18:36:51 on dist-test-slave-1xrh"
server_key: "3aae3b8c41c1da2cab58e82a12f2c908"
server_key_iv: "caba3493fde26304f548f1862222dadb"
server_key_version: "encryptionkey@0"
I20260806 18:36:51.613090 2283 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786041246094603-20232-0/minicluster-data/ts-1/wal/instance:
uuid: "61d7bb0f87d04b1b85aa1dbf30e6bc74"
format_stamp: "Formatted at 2026-08-06 18:36:51 on dist-test-slave-1xrh"
server_key: "3aae3b8c41c1da2cab58e82a12f2c908"
server_key_iv: "caba3493fde26304f548f1862222dadb"
server_key_version: "encryptionkey@0"
I20260806 18:36:51.618165 2283 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260806 18:36:51.620796 2298 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 18:36:51.622149 2283 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260806 18:36:51.622278 2283 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786041246094603-20232-0/minicluster-data/ts-1/data,/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786041246094603-20232-0/minicluster-data/ts-1/wal
uuid: "61d7bb0f87d04b1b85aa1dbf30e6bc74"
format_stamp: "Formatted at 2026-08-06 18:36:51 on dist-test-slave-1xrh"
server_key: "3aae3b8c41c1da2cab58e82a12f2c908"
server_key_iv: "caba3493fde26304f548f1862222dadb"
server_key_version: "encryptionkey@0"
I20260806 18:36:51.622400 2283 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786041246094603-20232-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786041246094603-20232-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786041246094603-20232-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 18:36:51.645401 2283 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 18:36:51.648593 2283 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 18:36:51.648793 2283 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 18:36:51.649397 2283 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 18:36:51.650254 2283 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 18:36:51.650323 2283 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 18:36:51.650415 2283 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 18:36:51.650466 2283 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 18:36:51.658980 2283 rpc_server.cc:307] RPC server started. Bound to: 127.19.194.2:40079
I20260806 18:36:51.659014 2420 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.194.2:40079 every 8 connection(s)
I20260806 18:36:51.659852 2283 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786041246094603-20232-0/minicluster-data/ts-1/data/info.pb
I20260806 18:36:51.666993 20232 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJW69tn/build/debug/bin/kudu as pid 2283
I20260806 18:36:51.667119 20232 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786041246094603-20232-0/minicluster-data/ts-1/wal/instance
I20260806 18:36:51.667472 20232 external_mini_cluster.cc:1428] Setting key 108411a66bebf0068172c20038d8e322
Aug 06 18:36:51 dist-test-slave-1xrh krb5kdc[2033](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041411, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.194.2@KRBTEST.COM for kudu/127.19.194.62@KRBTEST.COM
I20260806 18:36:51.675009 2122 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:36:51.661497 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.194.2:48381 (local address 127.19.194.62:40989)
0806 18:36:51.661626 (+ 129us) server_negotiation.cc:206] Beginning negotiation
0806 18:36:51.661630 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 18:36:51.662452 (+ 822us) server_negotiation.cc:407] Connection header received
0806 18:36:51.663948 (+ 1496us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:36:51.663951 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:36:51.664013 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:36:51.664092 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:36:51.665157 (+ 1065us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:36:51.665697 (+ 540us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:36:51.666809 (+ 1112us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:36:51.667061 (+ 252us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:36:51.667190 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:36:51.670217 (+ 3027us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:36:51.670222 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:36:51.670226 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 18:36:51.670256 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:36:51.672120 (+ 1864us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:36:51.672800 (+ 680us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:36:51.672807 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:36:51.672812 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:36:51.672911 (+ 99us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:36:51.673288 (+ 377us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:36:51.673294 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:36:51.673299 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:36:51.673605 (+ 306us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:36:51.673711 (+ 106us) server_negotiation.cc:1035] Waiting for connection context
0806 18:36:51.674670 (+ 959us) server_negotiation.cc:299] Negotiation successful
0806 18:36:51.674805 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260806 18:36:51.675370 2423 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:36:51.661800 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.194.62:40989 (local address 127.19.194.2:48381)
0806 18:36:51.662279 (+ 479us) negotiation.cc:107] Waiting for socket to connect
0806 18:36:51.662338 (+ 59us) client_negotiation.cc:174] Beginning negotiation
0806 18:36:51.663656 (+ 1318us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:36:51.664248 (+ 592us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:36:51.664254 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:36:51.664576 (+ 322us) client_negotiation.cc:189] Negotiated authn=SASL
0806 18:36:51.665015 (+ 439us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:36:51.665027 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:36:51.665838 (+ 811us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:36:51.665841 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:36:51.666671 (+ 830us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:36:51.666681 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:36:51.667411 (+ 730us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:36:51.667414 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:36:51.667570 (+ 156us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:36:51.668181 (+ 611us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 18:36:51.668222 (+ 41us) client_negotiation.cc:656] calling sasl_client_start()
0806 18:36:51.670051 (+ 1829us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 18:36:51.672253 (+ 2202us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:36:51.672262 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:36:51.672275 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:36:51.672702 (+ 427us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:36:51.673008 (+ 306us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:36:51.673011 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:36:51.673012 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:36:51.673200 (+ 188us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:36:51.673723 (+ 523us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 18:36:51.673730 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 18:36:51.673993 (+ 263us) client_negotiation.cc:769] Sending connection context
0806 18:36:51.674190 (+ 197us) client_negotiation.cc:240] Negotiation successful
0806 18:36:51.674459 (+ 269us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":311,"thread_start_us":159,"threads_started":1}
I20260806 18:36:51.676507 2421 heartbeater.cc:347] Connected to a master server at 127.19.194.62:40989
I20260806 18:36:51.676769 2421 heartbeater.cc:464] Registering TS with master...
I20260806 18:36:51.677238 2421 heartbeater.cc:511] Master 127.19.194.62:40989 requested a full tablet report, sending...
I20260806 18:36:51.678298 2080 ts_manager.cc:194] Registered new tserver with Master: 61d7bb0f87d04b1b85aa1dbf30e6bc74 (127.19.194.2:40079)
WARNING: no policy specified for kudu/127.19.194.3@KRBTEST.COM; defaulting to no policy
I20260806 18:36:51.678869 2080 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.19.194.2@KRBTEST.COM'} at 127.19.194.2:48381
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.19.194.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.19.194.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786041246094603-20232-0/krb5kdc/kudu_127.19.194.3.keytab.
Entry for principal kudu/127.19.194.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786041246094603-20232-0/krb5kdc/kudu_127.19.194.3.keytab.
I20260806 18:36:51.698750 20232 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJW69tn/build/debug/bin/kudu
/tmp/dist-test-taskJW69tn/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786041246094603-20232-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786041246094603-20232-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786041246094603-20232-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786041246094603-20232-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.19.194.3:0
--local_ip_for_outbound_sockets=127.19.194.3
--webserver_interface=127.19.194.3
--webserver_port=0
--tserver_master_addrs=127.19.194.62:40989
--keytab_file=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786041246094603-20232-0/krb5kdc/kudu_127.19.194.3.keytab
--principal=kudu/127.19.194.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.19.194.20:38847
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786041246094603-20232-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786041246094603-20232-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786041246094603-20232-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 18:36:51.808524 2426 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 18:36:51.808786 2426 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 18:36:51.808877 2426 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 18:36:51.812613 2426 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 18:36:51.812657 2426 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 18:36:51.812707 2426 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.19.194.3
I20260806 18:36:51.817734 2426 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.194.20:38847
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786041246094603-20232-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786041246094603-20232-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786041246094603-20232-0/krb5kdc/kudu_127.19.194.3.keytab
--principal=kudu/127.19.194.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.19.194.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786041246094603-20232-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.19.194.3
--webserver_port=0
--tserver_master_addrs=127.19.194.62:40989
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2426
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.19.194.3
--log_dir=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786041246094603-20232-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 17:59:13 UTC on ccycloud.quasar-wvsnzx.root.comops.site
build id 4571028
I20260806 18:36:51.818684 2426 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 18:36:51.819686 2426 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 18:36:51.829160 2434 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:36:51.829133 2432 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 18:36:51.829303 2426 server_base.cc:1034] running on GCE node
W20260806 18:36:51.829209 2431 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 18:36:51.829813 2426 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 18:36:51.830446 2426 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 18:36:51.831630 2426 hybrid_clock.cc:648] HybridClock initialized: now 1786041411831606 us; error 36 us; skew 500 ppm
Aug 06 18:36:51 dist-test-slave-1xrh krb5kdc[2033](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041411, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.194.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 18:36:51.834360 2426 init.cc:377] Logged in from keytab as kudu/127.19.194.3@KRBTEST.COM (short username kudu)
I20260806 18:36:51.835264 2426 webserver.cc:460] Webserver started at http://127.19.194.3:43387/ using document root <none> and password file <none>
I20260806 18:36:51.835749 2426 fs_manager.cc:359] Metadata directory not provided
I20260806 18:36:51.835807 2426 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 18:36:51.835960 2426 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 18:36:51.837822 2426 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786041246094603-20232-0/minicluster-data/ts-2/data/instance:
uuid: "1f52680fa3a54bd2983fe38ae400b5d5"
format_stamp: "Formatted at 2026-08-06 18:36:51 on dist-test-slave-1xrh"
server_key: "21c572fb9bd202689166d5ee43308cb2"
server_key_iv: "4916fbff0e77d6686fd222feaf1e5332"
server_key_version: "encryptionkey@0"
I20260806 18:36:51.838311 2426 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786041246094603-20232-0/minicluster-data/ts-2/wal/instance:
uuid: "1f52680fa3a54bd2983fe38ae400b5d5"
format_stamp: "Formatted at 2026-08-06 18:36:51 on dist-test-slave-1xrh"
server_key: "21c572fb9bd202689166d5ee43308cb2"
server_key_iv: "4916fbff0e77d6686fd222feaf1e5332"
server_key_version: "encryptionkey@0"
I20260806 18:36:51.841836 2426 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.000s sys 0.006s
I20260806 18:36:51.844110 2441 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 18:36:51.845299 2426 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.001s
I20260806 18:36:51.845417 2426 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786041246094603-20232-0/minicluster-data/ts-2/data,/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786041246094603-20232-0/minicluster-data/ts-2/wal
uuid: "1f52680fa3a54bd2983fe38ae400b5d5"
format_stamp: "Formatted at 2026-08-06 18:36:51 on dist-test-slave-1xrh"
server_key: "21c572fb9bd202689166d5ee43308cb2"
server_key_iv: "4916fbff0e77d6686fd222feaf1e5332"
server_key_version: "encryptionkey@0"
I20260806 18:36:51.845518 2426 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786041246094603-20232-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786041246094603-20232-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786041246094603-20232-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 18:36:51.857779 2426 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 18:36:51.860504 2426 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 18:36:51.860700 2426 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 18:36:51.861248 2426 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 18:36:51.862007 2426 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 18:36:51.862052 2426 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 18:36:51.862112 2426 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 18:36:51.862146 2426 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 18:36:51.870900 2426 rpc_server.cc:307] RPC server started. Bound to: 127.19.194.3:41377
I20260806 18:36:51.870924 2562 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.194.3:41377 every 8 connection(s)
I20260806 18:36:51.871901 2426 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786041246094603-20232-0/minicluster-data/ts-2/data/info.pb
I20260806 18:36:51.875026 20232 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJW69tn/build/debug/bin/kudu as pid 2426
I20260806 18:36:51.875178 20232 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786041246094603-20232-0/minicluster-data/ts-2/wal/instance
I20260806 18:36:51.875468 20232 external_mini_cluster.cc:1428] Setting key 0bef58d1b1f82842bb4cffc4691aa698
Aug 06 18:36:51 dist-test-slave-1xrh krb5kdc[2033](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041411, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.194.3@KRBTEST.COM for kudu/127.19.194.62@KRBTEST.COM
I20260806 18:36:51.885071 2122 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:36:51.873741 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.194.3:39761 (local address 127.19.194.62:40989)
0806 18:36:51.873878 (+ 137us) server_negotiation.cc:206] Beginning negotiation
0806 18:36:51.873882 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 18:36:51.874664 (+ 782us) server_negotiation.cc:407] Connection header received
0806 18:36:51.876272 (+ 1608us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:36:51.876275 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:36:51.876338 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:36:51.876443 (+ 105us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:36:51.877579 (+ 1136us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:36:51.878121 (+ 542us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:36:51.879271 (+ 1150us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:36:51.879493 (+ 222us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:36:51.879614 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:36:51.881935 (+ 2321us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:36:51.881938 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:36:51.881940 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 18:36:51.881964 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:36:51.883328 (+ 1364us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:36:51.883826 (+ 498us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:36:51.883829 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:36:51.883830 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:36:51.883904 (+ 74us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:36:51.884198 (+ 294us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:36:51.884200 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:36:51.884202 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:36:51.884368 (+ 166us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:36:51.884442 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0806 18:36:51.884852 (+ 410us) server_negotiation.cc:299] Negotiation successful
0806 18:36:51.884945 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260806 18:36:51.885857 2565 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:36:51.874060 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.194.62:40989 (local address 127.19.194.3:39761)
0806 18:36:51.874499 (+ 439us) negotiation.cc:107] Waiting for socket to connect
0806 18:36:51.874551 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0806 18:36:51.876062 (+ 1511us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:36:51.876638 (+ 576us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:36:51.876645 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:36:51.876984 (+ 339us) client_negotiation.cc:189] Negotiated authn=SASL
0806 18:36:51.877414 (+ 430us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:36:51.877427 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:36:51.878254 (+ 827us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:36:51.878258 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:36:51.879115 (+ 857us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:36:51.879147 (+ 32us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:36:51.879650 (+ 503us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:36:51.879655 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:36:51.879820 (+ 165us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:36:51.880319 (+ 499us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 18:36:51.880334 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0806 18:36:51.881744 (+ 1410us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 18:36:51.883445 (+ 1701us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:36:51.883449 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:36:51.883460 (+ 11us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:36:51.883728 (+ 268us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:36:51.883986 (+ 258us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:36:51.883988 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:36:51.883990 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:36:51.884117 (+ 127us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:36:51.884455 (+ 338us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 18:36:51.884460 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 18:36:51.884660 (+ 200us) client_negotiation.cc:769] Sending connection context
0806 18:36:51.884804 (+ 144us) client_negotiation.cc:240] Negotiation successful
0806 18:36:51.885088 (+ 284us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":249,"thread_start_us":110,"threads_started":1}
I20260806 18:36:51.886901 2563 heartbeater.cc:347] Connected to a master server at 127.19.194.62:40989
I20260806 18:36:51.887182 2563 heartbeater.cc:464] Registering TS with master...
I20260806 18:36:51.887662 2563 heartbeater.cc:511] Master 127.19.194.62:40989 requested a full tablet report, sending...
I20260806 18:36:51.888680 2080 ts_manager.cc:194] Registered new tserver with Master: 1f52680fa3a54bd2983fe38ae400b5d5 (127.19.194.3:41377)
I20260806 18:36:51.889227 2080 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.19.194.3@KRBTEST.COM'} at 127.19.194.3:39761
I20260806 18:36:51.889276 20232 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260806 18:36:51.898272 2122 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:36:51.891347 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59906 (local address 127.19.194.62:40989)
0806 18:36:51.891488 (+ 141us) server_negotiation.cc:206] Beginning negotiation
0806 18:36:51.891492 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 18:36:51.891677 (+ 185us) server_negotiation.cc:407] Connection header received
0806 18:36:51.891760 (+ 83us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:36:51.891762 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:36:51.891797 (+ 35us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:36:51.891871 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:36:51.892587 (+ 716us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:36:51.893075 (+ 488us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:36:51.893873 (+ 798us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:36:51.894076 (+ 203us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:36:51.894157 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:36:51.895472 (+ 1315us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:36:51.895475 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:36:51.895477 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 18:36:51.895498 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:36:51.896680 (+ 1182us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:36:51.897052 (+ 372us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:36:51.897054 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:36:51.897055 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:36:51.897129 (+ 74us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:36:51.897464 (+ 335us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:36:51.897468 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:36:51.897473 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:36:51.897683 (+ 210us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:36:51.897783 (+ 100us) server_negotiation.cc:1035] Waiting for connection context
0806 18:36:51.898000 (+ 217us) server_negotiation.cc:299] Negotiation successful
0806 18:36:51.898086 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260806 18:36:51.900866 2080 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:59906:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260806 18:36:51.903803 2080 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 18:36:51.918218 2578 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:36:51.912906 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50858 (local address 127.19.194.2:40079)
0806 18:36:51.913503 (+ 597us) server_negotiation.cc:206] Beginning negotiation
0806 18:36:51.913515 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0806 18:36:51.913550 (+ 35us) server_negotiation.cc:407] Connection header received
0806 18:36:51.913624 (+ 74us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:36:51.913629 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:36:51.913808 (+ 179us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:36:51.914224 (+ 416us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 18:36:51.914375 (+ 151us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:36:51.915456 (+ 1081us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:36:51.916899 (+ 1443us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:36:51.917464 (+ 565us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:36:51.917592 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:36:51.917741 (+ 149us) server_negotiation.cc:1035] Waiting for connection context
0806 18:36:51.917933 (+ 192us) server_negotiation.cc:299] Negotiation successful
0806 18:36:51.918044 (+ 111us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":497,"thread_start_us":111,"threads_started":1}
I20260806 18:36:51.918236 2576 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:36:51.912528 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.194.2:40079 (local address 127.0.0.1:50858)
0806 18:36:51.912999 (+ 471us) negotiation.cc:107] Waiting for socket to connect
0806 18:36:51.913027 (+ 28us) client_negotiation.cc:174] Beginning negotiation
0806 18:36:51.913117 (+ 90us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:36:51.913963 (+ 846us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:36:51.913968 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:36:51.913989 (+ 21us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 18:36:51.914076 (+ 87us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:36:51.914082 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:36:51.915588 (+ 1506us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:36:51.915591 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:36:51.916753 (+ 1162us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:36:51.916760 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:36:51.917593 (+ 833us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:36:51.917596 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:36:51.917719 (+ 123us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:36:51.917760 (+ 41us) client_negotiation.cc:769] Sending connection context
0806 18:36:51.917902 (+ 142us) client_negotiation.cc:240] Negotiation successful
0806 18:36:51.918071 (+ 169us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":412,"mutex_wait_us":73,"thread_start_us":80,"threads_started":1}
I20260806 18:36:51.918270 2574 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:36:51.912120 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.194.3:41377 (local address 127.0.0.1:47596)
0806 18:36:51.912464 (+ 344us) negotiation.cc:107] Waiting for socket to connect
0806 18:36:51.912515 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0806 18:36:51.912643 (+ 128us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:36:51.913471 (+ 828us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:36:51.913477 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:36:51.913502 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 18:36:51.913629 (+ 127us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:36:51.913636 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:36:51.915255 (+ 1619us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:36:51.915258 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:36:51.916741 (+ 1483us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:36:51.916753 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:36:51.917752 (+ 999us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:36:51.917756 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:36:51.917844 (+ 88us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:36:51.917871 (+ 27us) client_negotiation.cc:769] Sending connection context
0806 18:36:51.917926 (+ 55us) client_negotiation.cc:240] Negotiation successful
0806 18:36:51.918061 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":159,"mutex_wait_us":6,"thread_start_us":86,"threads_started":1}
I20260806 18:36:51.918633 2579 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:36:51.913419 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43014 (local address 127.19.194.1:42955)
0806 18:36:51.913832 (+ 413us) server_negotiation.cc:206] Beginning negotiation
0806 18:36:51.913849 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0806 18:36:51.913872 (+ 23us) server_negotiation.cc:407] Connection header received
0806 18:36:51.913984 (+ 112us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:36:51.913988 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:36:51.914116 (+ 128us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:36:51.914270 (+ 154us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 18:36:51.914450 (+ 180us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:36:51.915327 (+ 877us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:36:51.916753 (+ 1426us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:36:51.917587 (+ 834us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:36:51.917710 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:36:51.917852 (+ 142us) server_negotiation.cc:1035] Waiting for connection context
0806 18:36:51.918344 (+ 492us) server_negotiation.cc:299] Negotiation successful
0806 18:36:51.918449 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":301,"thread_start_us":117,"threads_started":1}
I20260806 18:36:51.919018 2575 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:36:51.912254 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47596 (local address 127.19.194.3:41377)
0806 18:36:51.912944 (+ 690us) server_negotiation.cc:206] Beginning negotiation
0806 18:36:51.912958 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0806 18:36:51.912984 (+ 26us) server_negotiation.cc:407] Connection header received
0806 18:36:51.913078 (+ 94us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:36:51.913084 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:36:51.913285 (+ 201us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:36:51.913778 (+ 493us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 18:36:51.913941 (+ 163us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:36:51.915085 (+ 1144us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:36:51.916871 (+ 1786us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:36:51.917619 (+ 748us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:36:51.918565 (+ 946us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:36:51.918700 (+ 135us) server_negotiation.cc:1035] Waiting for connection context
0806 18:36:51.918754 (+ 54us) server_negotiation.cc:299] Negotiation successful
0806 18:36:51.918871 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":588,"thread_start_us":143,"threads_started":1}
I20260806 18:36:51.918277 2577 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:36:51.912829 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.194.1:42955 (local address 127.0.0.1:43014)
0806 18:36:51.913727 (+ 898us) negotiation.cc:107] Waiting for socket to connect
0806 18:36:51.913741 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0806 18:36:51.913863 (+ 122us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:36:51.914248 (+ 385us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:36:51.914251 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:36:51.914269 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 18:36:51.914357 (+ 88us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:36:51.914363 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:36:51.915466 (+ 1103us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:36:51.915471 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:36:51.916627 (+ 1156us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:36:51.916637 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:36:51.917912 (+ 1275us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:36:51.917918 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:36:51.918003 (+ 85us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:36:51.918022 (+ 19us) client_negotiation.cc:769] Sending connection context
0806 18:36:51.918088 (+ 66us) client_negotiation.cc:240] Negotiation successful
0806 18:36:51.918150 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":834,"mutex_wait_us":56,"thread_start_us":104,"threads_started":1}
I20260806 18:36:51.921497 2493 tablet_service.cc:1467] Processing CreateTablet for tablet 5983348849dd46c9b7b30bda1044aa9d (DEFAULT_TABLE table=test-table [id=386534c1b94948cd97b99a6eed781718]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 18:36:51.921398 2350 tablet_service.cc:1467] Processing CreateTablet for tablet 5983348849dd46c9b7b30bda1044aa9d (DEFAULT_TABLE table=test-table [id=386534c1b94948cd97b99a6eed781718]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 18:36:51.921597 2210 tablet_service.cc:1467] Processing CreateTablet for tablet 5983348849dd46c9b7b30bda1044aa9d (DEFAULT_TABLE table=test-table [id=386534c1b94948cd97b99a6eed781718]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 18:36:51.922420 2493 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5983348849dd46c9b7b30bda1044aa9d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 18:36:51.922449 2350 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5983348849dd46c9b7b30bda1044aa9d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 18:36:51.922448 2210 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5983348849dd46c9b7b30bda1044aa9d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 18:36:51.929996 2581 tablet_bootstrap.cc:492] T 5983348849dd46c9b7b30bda1044aa9d P e5ac26f8baab47dd8096af286ff02861: Bootstrap starting.
I20260806 18:36:51.930040 2580 tablet_bootstrap.cc:492] T 5983348849dd46c9b7b30bda1044aa9d P 1f52680fa3a54bd2983fe38ae400b5d5: Bootstrap starting.
I20260806 18:36:51.931419 2582 tablet_bootstrap.cc:492] T 5983348849dd46c9b7b30bda1044aa9d P 61d7bb0f87d04b1b85aa1dbf30e6bc74: Bootstrap starting.
I20260806 18:36:51.932441 2581 tablet_bootstrap.cc:654] T 5983348849dd46c9b7b30bda1044aa9d P e5ac26f8baab47dd8096af286ff02861: Neither blocks nor log segments found. Creating new log.
I20260806 18:36:51.932441 2580 tablet_bootstrap.cc:654] T 5983348849dd46c9b7b30bda1044aa9d P 1f52680fa3a54bd2983fe38ae400b5d5: Neither blocks nor log segments found. Creating new log.
I20260806 18:36:51.933522 2582 tablet_bootstrap.cc:654] T 5983348849dd46c9b7b30bda1044aa9d P 61d7bb0f87d04b1b85aa1dbf30e6bc74: Neither blocks nor log segments found. Creating new log.
I20260806 18:36:51.933828 2581 log.cc:824] T 5983348849dd46c9b7b30bda1044aa9d P e5ac26f8baab47dd8096af286ff02861: Log is configured to *not* fsync() on all Append() calls
I20260806 18:36:51.933826 2580 log.cc:824] T 5983348849dd46c9b7b30bda1044aa9d P 1f52680fa3a54bd2983fe38ae400b5d5: Log is configured to *not* fsync() on all Append() calls
I20260806 18:36:51.934337 2582 log.cc:824] T 5983348849dd46c9b7b30bda1044aa9d P 61d7bb0f87d04b1b85aa1dbf30e6bc74: Log is configured to *not* fsync() on all Append() calls
I20260806 18:36:51.935854 2580 tablet_bootstrap.cc:492] T 5983348849dd46c9b7b30bda1044aa9d P 1f52680fa3a54bd2983fe38ae400b5d5: No bootstrap required, opened a new log
I20260806 18:36:51.935854 2581 tablet_bootstrap.cc:492] T 5983348849dd46c9b7b30bda1044aa9d P e5ac26f8baab47dd8096af286ff02861: No bootstrap required, opened a new log
I20260806 18:36:51.936043 2580 ts_tablet_manager.cc:1397] T 5983348849dd46c9b7b30bda1044aa9d P 1f52680fa3a54bd2983fe38ae400b5d5: Time spent bootstrapping tablet: real 0.006s user 0.005s sys 0.000s
I20260806 18:36:51.936043 2581 ts_tablet_manager.cc:1397] T 5983348849dd46c9b7b30bda1044aa9d P e5ac26f8baab47dd8096af286ff02861: Time spent bootstrapping tablet: real 0.006s user 0.001s sys 0.004s
I20260806 18:36:51.936302 2582 tablet_bootstrap.cc:492] T 5983348849dd46c9b7b30bda1044aa9d P 61d7bb0f87d04b1b85aa1dbf30e6bc74: No bootstrap required, opened a new log
I20260806 18:36:51.936496 2582 ts_tablet_manager.cc:1397] T 5983348849dd46c9b7b30bda1044aa9d P 61d7bb0f87d04b1b85aa1dbf30e6bc74: Time spent bootstrapping tablet: real 0.005s user 0.003s sys 0.000s
I20260806 18:36:51.939245 2582 raft_consensus.cc:348] T 5983348849dd46c9b7b30bda1044aa9d P 61d7bb0f87d04b1b85aa1dbf30e6bc74 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f52680fa3a54bd2983fe38ae400b5d5" member_type: VOTER last_known_addr { host: "127.19.194.3" port: 41377 } } peers { permanent_uuid: "e5ac26f8baab47dd8096af286ff02861" member_type: VOTER last_known_addr { host: "127.19.194.1" port: 42955 } } peers { permanent_uuid: "61d7bb0f87d04b1b85aa1dbf30e6bc74" member_type: VOTER last_known_addr { host: "127.19.194.2" port: 40079 } }
I20260806 18:36:51.939469 2582 raft_consensus.cc:374] T 5983348849dd46c9b7b30bda1044aa9d P 61d7bb0f87d04b1b85aa1dbf30e6bc74 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 18:36:51.939522 2582 raft_consensus.cc:729] T 5983348849dd46c9b7b30bda1044aa9d P 61d7bb0f87d04b1b85aa1dbf30e6bc74 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 61d7bb0f87d04b1b85aa1dbf30e6bc74, State: Initialized, Role: FOLLOWER
I20260806 18:36:51.939834 2581 raft_consensus.cc:348] T 5983348849dd46c9b7b30bda1044aa9d P e5ac26f8baab47dd8096af286ff02861 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f52680fa3a54bd2983fe38ae400b5d5" member_type: VOTER last_known_addr { host: "127.19.194.3" port: 41377 } } peers { permanent_uuid: "e5ac26f8baab47dd8096af286ff02861" member_type: VOTER last_known_addr { host: "127.19.194.1" port: 42955 } } peers { permanent_uuid: "61d7bb0f87d04b1b85aa1dbf30e6bc74" member_type: VOTER last_known_addr { host: "127.19.194.2" port: 40079 } }
I20260806 18:36:51.939829 2580 raft_consensus.cc:348] T 5983348849dd46c9b7b30bda1044aa9d P 1f52680fa3a54bd2983fe38ae400b5d5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f52680fa3a54bd2983fe38ae400b5d5" member_type: VOTER last_known_addr { host: "127.19.194.3" port: 41377 } } peers { permanent_uuid: "e5ac26f8baab47dd8096af286ff02861" member_type: VOTER last_known_addr { host: "127.19.194.1" port: 42955 } } peers { permanent_uuid: "61d7bb0f87d04b1b85aa1dbf30e6bc74" member_type: VOTER last_known_addr { host: "127.19.194.2" port: 40079 } }
I20260806 18:36:51.939937 2582 consensus_queue.cc:260] T 5983348849dd46c9b7b30bda1044aa9d P 61d7bb0f87d04b1b85aa1dbf30e6bc74 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f52680fa3a54bd2983fe38ae400b5d5" member_type: VOTER last_known_addr { host: "127.19.194.3" port: 41377 } } peers { permanent_uuid: "e5ac26f8baab47dd8096af286ff02861" member_type: VOTER last_known_addr { host: "127.19.194.1" port: 42955 } } peers { permanent_uuid: "61d7bb0f87d04b1b85aa1dbf30e6bc74" member_type: VOTER last_known_addr { host: "127.19.194.2" port: 40079 } }
I20260806 18:36:51.940062 2581 raft_consensus.cc:374] T 5983348849dd46c9b7b30bda1044aa9d P e5ac26f8baab47dd8096af286ff02861 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 18:36:51.940062 2580 raft_consensus.cc:374] T 5983348849dd46c9b7b30bda1044aa9d P 1f52680fa3a54bd2983fe38ae400b5d5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 18:36:51.940119 2581 raft_consensus.cc:729] T 5983348849dd46c9b7b30bda1044aa9d P e5ac26f8baab47dd8096af286ff02861 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e5ac26f8baab47dd8096af286ff02861, State: Initialized, Role: FOLLOWER
I20260806 18:36:51.940119 2580 raft_consensus.cc:729] T 5983348849dd46c9b7b30bda1044aa9d P 1f52680fa3a54bd2983fe38ae400b5d5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1f52680fa3a54bd2983fe38ae400b5d5, State: Initialized, Role: FOLLOWER
I20260806 18:36:51.940526 2580 consensus_queue.cc:260] T 5983348849dd46c9b7b30bda1044aa9d P 1f52680fa3a54bd2983fe38ae400b5d5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f52680fa3a54bd2983fe38ae400b5d5" member_type: VOTER last_known_addr { host: "127.19.194.3" port: 41377 } } peers { permanent_uuid: "e5ac26f8baab47dd8096af286ff02861" member_type: VOTER last_known_addr { host: "127.19.194.1" port: 42955 } } peers { permanent_uuid: "61d7bb0f87d04b1b85aa1dbf30e6bc74" member_type: VOTER last_known_addr { host: "127.19.194.2" port: 40079 } }
I20260806 18:36:51.940526 2581 consensus_queue.cc:260] T 5983348849dd46c9b7b30bda1044aa9d P e5ac26f8baab47dd8096af286ff02861 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f52680fa3a54bd2983fe38ae400b5d5" member_type: VOTER last_known_addr { host: "127.19.194.3" port: 41377 } } peers { permanent_uuid: "e5ac26f8baab47dd8096af286ff02861" member_type: VOTER last_known_addr { host: "127.19.194.1" port: 42955 } } peers { permanent_uuid: "61d7bb0f87d04b1b85aa1dbf30e6bc74" member_type: VOTER last_known_addr { host: "127.19.194.2" port: 40079 } }
I20260806 18:36:51.941210 2278 heartbeater.cc:503] Master 127.19.194.62:40989 was elected leader, sending a full tablet report...
I20260806 18:36:51.941262 2582 ts_tablet_manager.cc:1428] T 5983348849dd46c9b7b30bda1044aa9d P 61d7bb0f87d04b1b85aa1dbf30e6bc74: Time spent starting tablet: real 0.005s user 0.006s sys 0.000s
I20260806 18:36:51.941442 2581 ts_tablet_manager.cc:1428] T 5983348849dd46c9b7b30bda1044aa9d P e5ac26f8baab47dd8096af286ff02861: Time spent starting tablet: real 0.005s user 0.006s sys 0.000s
I20260806 18:36:51.941442 2580 ts_tablet_manager.cc:1428] T 5983348849dd46c9b7b30bda1044aa9d P 1f52680fa3a54bd2983fe38ae400b5d5: Time spent starting tablet: real 0.005s user 0.002s sys 0.004s
I20260806 18:36:51.941541 2421 heartbeater.cc:503] Master 127.19.194.62:40989 was elected leader, sending a full tablet report...
I20260806 18:36:51.942386 2563 heartbeater.cc:503] Master 127.19.194.62:40989 was elected leader, sending a full tablet report...
W20260806 18:36:52.122970 2564 tablet.cc:2369] T 5983348849dd46c9b7b30bda1044aa9d P 1f52680fa3a54bd2983fe38ae400b5d5: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 18:36:52.158892 2587 raft_consensus.cc:482] T 5983348849dd46c9b7b30bda1044aa9d P e5ac26f8baab47dd8096af286ff02861 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 18:36:52.159108 2587 raft_consensus.cc:504] T 5983348849dd46c9b7b30bda1044aa9d P e5ac26f8baab47dd8096af286ff02861 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f52680fa3a54bd2983fe38ae400b5d5" member_type: VOTER last_known_addr { host: "127.19.194.3" port: 41377 } } peers { permanent_uuid: "e5ac26f8baab47dd8096af286ff02861" member_type: VOTER last_known_addr { host: "127.19.194.1" port: 42955 } } peers { permanent_uuid: "61d7bb0f87d04b1b85aa1dbf30e6bc74" member_type: VOTER last_known_addr { host: "127.19.194.2" port: 40079 } }
I20260806 18:36:52.160254 2587 leader_election.cc:288] T 5983348849dd46c9b7b30bda1044aa9d P e5ac26f8baab47dd8096af286ff02861 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1f52680fa3a54bd2983fe38ae400b5d5 (127.19.194.3:41377), 61d7bb0f87d04b1b85aa1dbf30e6bc74 (127.19.194.2:40079)
W20260806 18:36:52.160976 2422 tablet.cc:2369] T 5983348849dd46c9b7b30bda1044aa9d P 61d7bb0f87d04b1b85aa1dbf30e6bc74: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 18:36:52.164247 2578 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:36:52.160875 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.194.1:46365 (local address 127.19.194.2:40079)
0806 18:36:52.161201 (+ 326us) server_negotiation.cc:206] Beginning negotiation
0806 18:36:52.161205 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 18:36:52.161217 (+ 12us) server_negotiation.cc:407] Connection header received
0806 18:36:52.161268 (+ 51us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:36:52.161273 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:36:52.161319 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:36:52.161411 (+ 92us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 18:36:52.161709 (+ 298us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:36:52.162209 (+ 500us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:36:52.163300 (+ 1091us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:36:52.163682 (+ 382us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:36:52.163826 (+ 144us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:36:52.163883 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0806 18:36:52.164069 (+ 186us) server_negotiation.cc:299] Negotiation successful
0806 18:36:52.164133 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":232}
I20260806 18:36:52.164268 2589 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:36:52.160490 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.194.2:40079 (local address 127.19.194.1:46365)
0806 18:36:52.160973 (+ 483us) negotiation.cc:107] Waiting for socket to connect
0806 18:36:52.160989 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0806 18:36:52.161090 (+ 101us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:36:52.161428 (+ 338us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:36:52.161431 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:36:52.161447 (+ 16us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 18:36:52.161591 (+ 144us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:36:52.161596 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:36:52.162325 (+ 729us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:36:52.162329 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:36:52.163175 (+ 846us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:36:52.163192 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:36:52.163819 (+ 627us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:36:52.163823 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:36:52.163930 (+ 107us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:36:52.163958 (+ 28us) client_negotiation.cc:769] Sending connection context
0806 18:36:52.164053 (+ 95us) client_negotiation.cc:240] Negotiation successful
0806 18:36:52.164123 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":399,"thread_start_us":148,"threads_started":1}
I20260806 18:36:52.164317 2575 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:36:52.160624 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.194.1:60123 (local address 127.19.194.3:41377)
0806 18:36:52.160779 (+ 155us) server_negotiation.cc:206] Beginning negotiation
0806 18:36:52.160784 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 18:36:52.161027 (+ 243us) server_negotiation.cc:407] Connection header received
0806 18:36:52.161240 (+ 213us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:36:52.161244 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:36:52.161295 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:36:52.161340 (+ 45us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 18:36:52.161789 (+ 449us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:36:52.162272 (+ 483us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:36:52.163342 (+ 1070us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:36:52.163736 (+ 394us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:36:52.163888 (+ 152us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:36:52.163943 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0806 18:36:52.164141 (+ 198us) server_negotiation.cc:299] Negotiation successful
0806 18:36:52.164210 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260806 18:36:52.164335 2590 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:36:52.160455 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.194.3:41377 (local address 127.19.194.1:60123)
0806 18:36:52.160935 (+ 480us) negotiation.cc:107] Waiting for socket to connect
0806 18:36:52.160949 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0806 18:36:52.161108 (+ 159us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:36:52.161503 (+ 395us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:36:52.161506 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:36:52.161542 (+ 36us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 18:36:52.161655 (+ 113us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:36:52.161660 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:36:52.162388 (+ 728us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:36:52.162395 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:36:52.163216 (+ 821us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:36:52.163226 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:36:52.163883 (+ 657us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:36:52.163887 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:36:52.164010 (+ 123us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:36:52.164030 (+ 20us) client_negotiation.cc:769] Sending connection context
0806 18:36:52.164122 (+ 92us) client_negotiation.cc:240] Negotiation successful
0806 18:36:52.164191 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":411,"thread_start_us":89,"threads_started":1}
I20260806 18:36:52.164841 2372 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5983348849dd46c9b7b30bda1044aa9d" candidate_uuid: "e5ac26f8baab47dd8096af286ff02861" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "61d7bb0f87d04b1b85aa1dbf30e6bc74" is_pre_election: true
I20260806 18:36:52.164868 2514 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5983348849dd46c9b7b30bda1044aa9d" candidate_uuid: "e5ac26f8baab47dd8096af286ff02861" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1f52680fa3a54bd2983fe38ae400b5d5" is_pre_election: true
I20260806 18:36:52.165110 2372 raft_consensus.cc:2454] T 5983348849dd46c9b7b30bda1044aa9d P 61d7bb0f87d04b1b85aa1dbf30e6bc74 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e5ac26f8baab47dd8096af286ff02861 in term 0.
I20260806 18:36:52.165153 2514 raft_consensus.cc:2454] T 5983348849dd46c9b7b30bda1044aa9d P 1f52680fa3a54bd2983fe38ae400b5d5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e5ac26f8baab47dd8096af286ff02861 in term 0.
I20260806 18:36:52.165552 2162 leader_election.cc:302] T 5983348849dd46c9b7b30bda1044aa9d P e5ac26f8baab47dd8096af286ff02861 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 61d7bb0f87d04b1b85aa1dbf30e6bc74, e5ac26f8baab47dd8096af286ff02861; no voters:
I20260806 18:36:52.165820 2587 raft_consensus.cc:2781] T 5983348849dd46c9b7b30bda1044aa9d P e5ac26f8baab47dd8096af286ff02861 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 18:36:52.165910 2587 raft_consensus.cc:482] T 5983348849dd46c9b7b30bda1044aa9d P e5ac26f8baab47dd8096af286ff02861 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 18:36:52.165956 2587 raft_consensus.cc:3037] T 5983348849dd46c9b7b30bda1044aa9d P e5ac26f8baab47dd8096af286ff02861 [term 0 FOLLOWER]: Advancing to term 1
I20260806 18:36:52.166953 2587 raft_consensus.cc:504] T 5983348849dd46c9b7b30bda1044aa9d P e5ac26f8baab47dd8096af286ff02861 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f52680fa3a54bd2983fe38ae400b5d5" member_type: VOTER last_known_addr { host: "127.19.194.3" port: 41377 } } peers { permanent_uuid: "e5ac26f8baab47dd8096af286ff02861" member_type: VOTER last_known_addr { host: "127.19.194.1" port: 42955 } } peers { permanent_uuid: "61d7bb0f87d04b1b85aa1dbf30e6bc74" member_type: VOTER last_known_addr { host: "127.19.194.2" port: 40079 } }
I20260806 18:36:52.167333 2587 leader_election.cc:288] T 5983348849dd46c9b7b30bda1044aa9d P e5ac26f8baab47dd8096af286ff02861 [CANDIDATE]: Term 1 election: Requested vote from peers 1f52680fa3a54bd2983fe38ae400b5d5 (127.19.194.3:41377), 61d7bb0f87d04b1b85aa1dbf30e6bc74 (127.19.194.2:40079)
I20260806 18:36:52.167636 2514 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5983348849dd46c9b7b30bda1044aa9d" candidate_uuid: "e5ac26f8baab47dd8096af286ff02861" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1f52680fa3a54bd2983fe38ae400b5d5"
I20260806 18:36:52.167666 2372 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5983348849dd46c9b7b30bda1044aa9d" candidate_uuid: "e5ac26f8baab47dd8096af286ff02861" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "61d7bb0f87d04b1b85aa1dbf30e6bc74"
I20260806 18:36:52.167771 2514 raft_consensus.cc:3037] T 5983348849dd46c9b7b30bda1044aa9d P 1f52680fa3a54bd2983fe38ae400b5d5 [term 0 FOLLOWER]: Advancing to term 1
I20260806 18:36:52.167784 2372 raft_consensus.cc:3037] T 5983348849dd46c9b7b30bda1044aa9d P 61d7bb0f87d04b1b85aa1dbf30e6bc74 [term 0 FOLLOWER]: Advancing to term 1
I20260806 18:36:52.168823 2372 raft_consensus.cc:2454] T 5983348849dd46c9b7b30bda1044aa9d P 61d7bb0f87d04b1b85aa1dbf30e6bc74 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e5ac26f8baab47dd8096af286ff02861 in term 1.
I20260806 18:36:52.168826 2514 raft_consensus.cc:2454] T 5983348849dd46c9b7b30bda1044aa9d P 1f52680fa3a54bd2983fe38ae400b5d5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e5ac26f8baab47dd8096af286ff02861 in term 1.
I20260806 18:36:52.169122 2160 leader_election.cc:302] T 5983348849dd46c9b7b30bda1044aa9d P e5ac26f8baab47dd8096af286ff02861 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1f52680fa3a54bd2983fe38ae400b5d5, e5ac26f8baab47dd8096af286ff02861; no voters:
I20260806 18:36:52.169312 2587 raft_consensus.cc:2781] T 5983348849dd46c9b7b30bda1044aa9d P e5ac26f8baab47dd8096af286ff02861 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 18:36:52.169502 2587 raft_consensus.cc:686] T 5983348849dd46c9b7b30bda1044aa9d P e5ac26f8baab47dd8096af286ff02861 [term 1 LEADER]: Becoming Leader. State: Replica: e5ac26f8baab47dd8096af286ff02861, State: Running, Role: LEADER
I20260806 18:36:52.169829 2587 consensus_queue.cc:237] T 5983348849dd46c9b7b30bda1044aa9d P e5ac26f8baab47dd8096af286ff02861 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f52680fa3a54bd2983fe38ae400b5d5" member_type: VOTER last_known_addr { host: "127.19.194.3" port: 41377 } } peers { permanent_uuid: "e5ac26f8baab47dd8096af286ff02861" member_type: VOTER last_known_addr { host: "127.19.194.1" port: 42955 } } peers { permanent_uuid: "61d7bb0f87d04b1b85aa1dbf30e6bc74" member_type: VOTER last_known_addr { host: "127.19.194.2" port: 40079 } }
I20260806 18:36:52.173053 2079 catalog_manager.cc:5495] T 5983348849dd46c9b7b30bda1044aa9d P e5ac26f8baab47dd8096af286ff02861 reported cstate change: term changed from 0 to 1, leader changed from <none> to e5ac26f8baab47dd8096af286ff02861 (127.19.194.1). New cstate: current_term: 1 leader_uuid: "e5ac26f8baab47dd8096af286ff02861" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f52680fa3a54bd2983fe38ae400b5d5" member_type: VOTER last_known_addr { host: "127.19.194.3" port: 41377 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e5ac26f8baab47dd8096af286ff02861" member_type: VOTER last_known_addr { host: "127.19.194.1" port: 42955 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "61d7bb0f87d04b1b85aa1dbf30e6bc74" member_type: VOTER last_known_addr { host: "127.19.194.2" port: 40079 } health_report { overall_health: UNKNOWN } } }
W20260806 18:36:52.183425 2279 tablet.cc:2369] T 5983348849dd46c9b7b30bda1044aa9d P e5ac26f8baab47dd8096af286ff02861: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 18:36:52.210048 2579 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:36:52.206555 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43026 (local address 127.19.194.1:42955)
0806 18:36:52.206693 (+ 138us) server_negotiation.cc:206] Beginning negotiation
0806 18:36:52.206697 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 18:36:52.206711 (+ 14us) server_negotiation.cc:407] Connection header received
0806 18:36:52.206851 (+ 140us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:36:52.206854 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:36:52.206906 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:36:52.206990 (+ 84us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 18:36:52.207233 (+ 243us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:36:52.207725 (+ 492us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:36:52.208593 (+ 868us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:36:52.208804 (+ 211us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:36:52.208920 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:36:52.209274 (+ 354us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 18:36:52.209657 (+ 383us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 18:36:52.209791 (+ 134us) server_negotiation.cc:1035] Waiting for connection context
0806 18:36:52.209863 (+ 72us) server_negotiation.cc:299] Negotiation successful
0806 18:36:52.209913 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260806 18:36:52.215476 2372 raft_consensus.cc:1261] T 5983348849dd46c9b7b30bda1044aa9d P 61d7bb0f87d04b1b85aa1dbf30e6bc74 [term 1 FOLLOWER]: Refusing update from remote peer e5ac26f8baab47dd8096af286ff02861: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 18:36:52.215494 2514 raft_consensus.cc:1261] T 5983348849dd46c9b7b30bda1044aa9d P 1f52680fa3a54bd2983fe38ae400b5d5 [term 1 FOLLOWER]: Refusing update from remote peer e5ac26f8baab47dd8096af286ff02861: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 18:36:52.216157 2587 consensus_queue.cc:1035] T 5983348849dd46c9b7b30bda1044aa9d P e5ac26f8baab47dd8096af286ff02861 [LEADER]: Connected to new peer: Peer: permanent_uuid: "61d7bb0f87d04b1b85aa1dbf30e6bc74" member_type: VOTER last_known_addr { host: "127.19.194.2" port: 40079 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 18:36:52.216368 2591 consensus_queue.cc:1035] T 5983348849dd46c9b7b30bda1044aa9d P e5ac26f8baab47dd8096af286ff02861 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1f52680fa3a54bd2983fe38ae400b5d5" member_type: VOTER last_known_addr { host: "127.19.194.3" port: 41377 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 18:36:52.223728 2596 mvcc.cc:205] Tried to move back new op lower bound from 7315625624427470848 to 7315625624250183680. Current Snapshot: MvccSnapshot[applied={T|T < 7315625624427470848}]
I20260806 18:36:52.226328 2595 mvcc.cc:205] Tried to move back new op lower bound from 7315625624427470848 to 7315625624250183680. Current Snapshot: MvccSnapshot[applied={T|T < 7315625624427470848}]
I20260806 18:36:52.231515 2079 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:59906:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 18:36:52.231690 2079 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:59906:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 18:36:52.234087 2079 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 422f52e3e25c47249416f4d5b21c77be: Sending DeleteTablet for 3 replicas of tablet 5983348849dd46c9b7b30bda1044aa9d
I20260806 18:36:52.234929 2350 tablet_service.cc:1514] Processing DeleteTablet for tablet 5983348849dd46c9b7b30bda1044aa9d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 18:36:52 UTC) from {username='kudu', principal='kudu/127.19.194.62@KRBTEST.COM'} at 127.0.0.1:50858
I20260806 18:36:52.234973 2493 tablet_service.cc:1514] Processing DeleteTablet for tablet 5983348849dd46c9b7b30bda1044aa9d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 18:36:52 UTC) from {username='kudu', principal='kudu/127.19.194.62@KRBTEST.COM'} at 127.0.0.1:47596
I20260806 18:36:52.235407 2604 tablet_replica.cc:330] T 5983348849dd46c9b7b30bda1044aa9d P 61d7bb0f87d04b1b85aa1dbf30e6bc74: stopping tablet replica
I20260806 18:36:52.235447 2210 tablet_service.cc:1514] Processing DeleteTablet for tablet 5983348849dd46c9b7b30bda1044aa9d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 18:36:52 UTC) from {username='kudu', principal='kudu/127.19.194.62@KRBTEST.COM'} at 127.0.0.1:43014
I20260806 18:36:52.235713 2604 raft_consensus.cc:2229] T 5983348849dd46c9b7b30bda1044aa9d P 61d7bb0f87d04b1b85aa1dbf30e6bc74 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 18:36:52.235939 2606 tablet_replica.cc:330] T 5983348849dd46c9b7b30bda1044aa9d P e5ac26f8baab47dd8096af286ff02861: stopping tablet replica
I20260806 18:36:52.236025 20232 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJW69tn/build/debug/bin/kudu with pid 2144
I20260806 18:36:52.236047 2604 raft_consensus.cc:2258] T 5983348849dd46c9b7b30bda1044aa9d P 61d7bb0f87d04b1b85aa1dbf30e6bc74 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 18:36:52.238569 2604 ts_tablet_manager.cc:1905] T 5983348849dd46c9b7b30bda1044aa9d P 61d7bb0f87d04b1b85aa1dbf30e6bc74: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 18:36:52.238879 2605 tablet_replica.cc:330] T 5983348849dd46c9b7b30bda1044aa9d P 1f52680fa3a54bd2983fe38ae400b5d5: stopping tablet replica
I20260806 18:36:52.239202 2605 raft_consensus.cc:2229] T 5983348849dd46c9b7b30bda1044aa9d P 1f52680fa3a54bd2983fe38ae400b5d5 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 18:36:52.239471 2605 raft_consensus.cc:2258] T 5983348849dd46c9b7b30bda1044aa9d P 1f52680fa3a54bd2983fe38ae400b5d5 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 18:36:52.240912 2605 ts_tablet_manager.cc:1905] T 5983348849dd46c9b7b30bda1044aa9d P 1f52680fa3a54bd2983fe38ae400b5d5: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 18:36:52.241041 2604 ts_tablet_manager.cc:1918] T 5983348849dd46c9b7b30bda1044aa9d P 61d7bb0f87d04b1b85aa1dbf30e6bc74: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 18:36:52.241173 2604 log.cc:1196] T 5983348849dd46c9b7b30bda1044aa9d P 61d7bb0f87d04b1b85aa1dbf30e6bc74: Deleting WAL directory at /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786041246094603-20232-0/minicluster-data/ts-1/wal/wals/5983348849dd46c9b7b30bda1044aa9d
I20260806 18:36:52.241472 2604 ts_tablet_manager.cc:1939] T 5983348849dd46c9b7b30bda1044aa9d P 61d7bb0f87d04b1b85aa1dbf30e6bc74: Deleting consensus metadata
I20260806 18:36:52.242517 2066 catalog_manager.cc:4841] TS 61d7bb0f87d04b1b85aa1dbf30e6bc74 (127.19.194.2:40079): tablet 5983348849dd46c9b7b30bda1044aa9d (table test-table [id=386534c1b94948cd97b99a6eed781718]) successfully deleted
I20260806 18:36:52.244395 2605 ts_tablet_manager.cc:1918] T 5983348849dd46c9b7b30bda1044aa9d P 1f52680fa3a54bd2983fe38ae400b5d5: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 18:36:52.244498 2605 log.cc:1196] T 5983348849dd46c9b7b30bda1044aa9d P 1f52680fa3a54bd2983fe38ae400b5d5: Deleting WAL directory at /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786041246094603-20232-0/minicluster-data/ts-2/wal/wals/5983348849dd46c9b7b30bda1044aa9d
I20260806 18:36:52.244846 2605 ts_tablet_manager.cc:1939] T 5983348849dd46c9b7b30bda1044aa9d P 1f52680fa3a54bd2983fe38ae400b5d5: Deleting consensus metadata
I20260806 18:36:52.246088 20232 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJW69tn/build/debug/bin/kudu with pid 2283
W20260806 18:36:52.246069 2064 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.19.194.1:42955 (error 108)
I20260806 18:36:52.246671 2064 catalog_manager.cc:4841] TS 1f52680fa3a54bd2983fe38ae400b5d5 (127.19.194.3:41377): tablet 5983348849dd46c9b7b30bda1044aa9d (table test-table [id=386534c1b94948cd97b99a6eed781718]) successfully deleted
I20260806 18:36:52.248394 2577 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:36:52.248092 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.194.1:42955 (local address 127.0.0.1:43034)
0806 18:36:52.248221 (+ 129us) negotiation.cc:107] Waiting for socket to connect
0806 18:36:52.248294 (+ 73us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.194.1:42955: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":50}
W20260806 18:36:52.248612 2064 catalog_manager.cc:4584] TS e5ac26f8baab47dd8096af286ff02861 (127.19.194.1:42955): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 5983348849dd46c9b7b30bda1044aa9d: Network error: Client connection negotiation failed: client connection to 127.19.194.1:42955: connect: Connection refused (error 111)
I20260806 18:36:52.254163 20232 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJW69tn/build/debug/bin/kudu with pid 2426
I20260806 18:36:52.260675 20232 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJW69tn/build/debug/bin/kudu with pid 2047
I20260806 18:36:52.267743 20232 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJW69tn/build/debug/bin/kudu
/tmp/dist-test-taskJW69tn/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786041246094603-20232-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786041246094603-20232-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786041246094603-20232-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786041246094603-20232-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.19.194.62:40989
--webserver_interface=127.19.194.62
--webserver_port=33889
--keytab_file=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786041246094603-20232-0/krb5kdc/kudu_127.19.194.62.keytab
--principal=kudu/127.19.194.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.19.194.20:38847
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.19.194.62:40989
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--ipki_private_key_password_cmd=echo foo with env {KRB5CCNAME=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786041246094603-20232-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786041246094603-20232-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786041246094603-20232-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 18:36:52.376204 2608 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 18:36:52.376444 2608 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 18:36:52.376487 2608 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 18:36:52.380386 2608 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 18:36:52.380438 2608 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 18:36:52.380458 2608 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 18:36:52.380474 2608 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 18:36:52.380489 2608 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 18:36:52.385532 2608 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.194.20:38847
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786041246094603-20232-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786041246094603-20232-0/minicluster-data/master-0/wal
--ipki_private_key_password_cmd=echo foo
--ipki_ca_key_size=768
--master_addresses=127.19.194.62:40989
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786041246094603-20232-0/krb5kdc/kudu_127.19.194.62.keytab
--principal=kudu/127.19.194.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.19.194.62:40989
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786041246094603-20232-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.19.194.62
--webserver_port=33889
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2608
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786041246094603-20232-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 17:59:13 UTC on ccycloud.quasar-wvsnzx.root.comops.site
build id 4571028
I20260806 18:36:52.386439 2608 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 18:36:52.387470 2608 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 18:36:52.401932 2614 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 18:36:52.402031 2608 server_base.cc:1034] running on GCE node
W20260806 18:36:52.401960 2617 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:36:52.401970 2615 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 18:36:52.402817 2608 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 18:36:52.404011 2608 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 18:36:52.405171 2608 hybrid_clock.cc:648] HybridClock initialized: now 1786041412405157 us; error 28 us; skew 500 ppm
Aug 06 18:36:52 dist-test-slave-1xrh krb5kdc[2033](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041412, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.194.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 18:36:52.408051 2608 init.cc:377] Logged in from keytab as kudu/127.19.194.62@KRBTEST.COM (short username kudu)
I20260806 18:36:52.408967 2608 webserver.cc:460] Webserver started at http://127.19.194.62:33889/ using document root <none> and password file <none>
I20260806 18:36:52.409608 2608 fs_manager.cc:359] Metadata directory not provided
I20260806 18:36:52.409705 2608 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 18:36:52.413553 2608 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.004s sys 0.000s
I20260806 18:36:52.415688 2625 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 18:36:52.416879 2608 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260806 18:36:52.417001 2608 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786041246094603-20232-0/minicluster-data/master-0/data,/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786041246094603-20232-0/minicluster-data/master-0/wal
uuid: "422f52e3e25c47249416f4d5b21c77be"
format_stamp: "Formatted at 2026-08-06 18:36:51 on dist-test-slave-1xrh"
server_key: "5a4c117b1fc3712dd62d3e9f64694880"
server_key_iv: "78c55a7345d150ac7b0dbed4e7f1b0c6"
server_key_version: "encryptionkey@0"
I20260806 18:36:52.417598 2608 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786041246094603-20232-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786041246094603-20232-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786041246094603-20232-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 18:36:52.440618 2608 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 18:36:52.443863 2608 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 18:36:52.444075 2608 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 18:36:52.451047 2608 rpc_server.cc:307] RPC server started. Bound to: 127.19.194.62:40989
I20260806 18:36:52.451370 2682 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.194.62:40989 every 8 connection(s)
I20260806 18:36:52.452116 2608 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786041246094603-20232-0/minicluster-data/master-0/data/info.pb
I20260806 18:36:52.453557 20232 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJW69tn/build/debug/bin/kudu as pid 2608
I20260806 18:36:52.453969 20232 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJW69tn/build/debug/bin/kudu
/tmp/dist-test-taskJW69tn/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786041246094603-20232-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786041246094603-20232-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786041246094603-20232-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786041246094603-20232-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.19.194.1:42955
--local_ip_for_outbound_sockets=127.19.194.1
--tserver_master_addrs=127.19.194.62:40989
--webserver_port=38357
--webserver_interface=127.19.194.1
--keytab_file=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786041246094603-20232-0/krb5kdc/kudu_127.19.194.1.keytab
--principal=kudu/127.19.194.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.19.194.20:38847
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786041246094603-20232-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786041246094603-20232-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786041246094603-20232-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260806 18:36:52.456382 2683 sys_catalog.cc:263] Verifying existing consensus state
I20260806 18:36:52.457942 2683 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 422f52e3e25c47249416f4d5b21c77be: Bootstrap starting.
I20260806 18:36:52.474778 2683 log.cc:824] T 00000000000000000000000000000000 P 422f52e3e25c47249416f4d5b21c77be: Log is configured to *not* fsync() on all Append() calls
I20260806 18:36:52.483289 2683 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 422f52e3e25c47249416f4d5b21c77be: Bootstrap replayed 1/1 log segments. Stats: ops{read=8 overwritten=0 applied=8 ignored=0} inserts{seen=5 ignored=0} mutations{seen=4 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260806 18:36:52.483672 2683 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 422f52e3e25c47249416f4d5b21c77be: Bootstrap complete.
I20260806 18:36:52.487740 2683 raft_consensus.cc:348] T 00000000000000000000000000000000 P 422f52e3e25c47249416f4d5b21c77be [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "422f52e3e25c47249416f4d5b21c77be" member_type: VOTER last_known_addr { host: "127.19.194.62" port: 40989 } }
I20260806 18:36:52.488150 2683 raft_consensus.cc:729] T 00000000000000000000000000000000 P 422f52e3e25c47249416f4d5b21c77be [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 422f52e3e25c47249416f4d5b21c77be, State: Initialized, Role: FOLLOWER
I20260806 18:36:52.488680 2683 consensus_queue.cc:260] T 00000000000000000000000000000000 P 422f52e3e25c47249416f4d5b21c77be [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "422f52e3e25c47249416f4d5b21c77be" member_type: VOTER last_known_addr { host: "127.19.194.62" port: 40989 } }
I20260806 18:36:52.488823 2683 raft_consensus.cc:388] T 00000000000000000000000000000000 P 422f52e3e25c47249416f4d5b21c77be [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 18:36:52.488926 2683 raft_consensus.cc:482] T 00000000000000000000000000000000 P 422f52e3e25c47249416f4d5b21c77be [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 18:36:52.489040 2683 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 422f52e3e25c47249416f4d5b21c77be [term 1 FOLLOWER]: Advancing to term 2
I20260806 18:36:52.491219 2683 raft_consensus.cc:504] T 00000000000000000000000000000000 P 422f52e3e25c47249416f4d5b21c77be [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "422f52e3e25c47249416f4d5b21c77be" member_type: VOTER last_known_addr { host: "127.19.194.62" port: 40989 } }
I20260806 18:36:52.491552 2683 leader_election.cc:302] T 00000000000000000000000000000000 P 422f52e3e25c47249416f4d5b21c77be [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 422f52e3e25c47249416f4d5b21c77be; no voters:
I20260806 18:36:52.491879 2683 leader_election.cc:288] T 00000000000000000000000000000000 P 422f52e3e25c47249416f4d5b21c77be [CANDIDATE]: Term 2 election: Requested vote from peers
I20260806 18:36:52.491988 2687 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 422f52e3e25c47249416f4d5b21c77be [term 2 FOLLOWER]: Leader election won for term 2
I20260806 18:36:52.492208 2687 raft_consensus.cc:686] T 00000000000000000000000000000000 P 422f52e3e25c47249416f4d5b21c77be [term 2 LEADER]: Becoming Leader. State: Replica: 422f52e3e25c47249416f4d5b21c77be, State: Running, Role: LEADER
I20260806 18:36:52.492498 2687 consensus_queue.cc:237] T 00000000000000000000000000000000 P 422f52e3e25c47249416f4d5b21c77be [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "422f52e3e25c47249416f4d5b21c77be" member_type: VOTER last_known_addr { host: "127.19.194.62" port: 40989 } }
I20260806 18:36:52.493074 2683 sys_catalog.cc:564] T 00000000000000000000000000000000 P 422f52e3e25c47249416f4d5b21c77be [sys.catalog]: configured and running, proceeding with master startup.
I20260806 18:36:52.493394 2687 sys_catalog.cc:455] T 00000000000000000000000000000000 P 422f52e3e25c47249416f4d5b21c77be [sys.catalog]: SysCatalogTable state changed. Reason: New leader 422f52e3e25c47249416f4d5b21c77be. Latest consensus state: current_term: 2 leader_uuid: "422f52e3e25c47249416f4d5b21c77be" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "422f52e3e25c47249416f4d5b21c77be" member_type: VOTER last_known_addr { host: "127.19.194.62" port: 40989 } } }
I20260806 18:36:52.493512 2687 sys_catalog.cc:458] T 00000000000000000000000000000000 P 422f52e3e25c47249416f4d5b21c77be [sys.catalog]: This master's current role is: LEADER
I20260806 18:36:52.493863 2691 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 18:36:52.493891 2688 sys_catalog.cc:455] T 00000000000000000000000000000000 P 422f52e3e25c47249416f4d5b21c77be [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "422f52e3e25c47249416f4d5b21c77be" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "422f52e3e25c47249416f4d5b21c77be" member_type: VOTER last_known_addr { host: "127.19.194.62" port: 40989 } } }
I20260806 18:36:52.494097 2688 sys_catalog.cc:458] T 00000000000000000000000000000000 P 422f52e3e25c47249416f4d5b21c77be [sys.catalog]: This master's current role is: LEADER
I20260806 18:36:52.498454 2691 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 18:36:52.499228 2691 catalog_manager.cc:1259] Loaded cluster ID: 4eef91136e8e4ae8b146e3827730dab6
I20260806 18:36:52.499289 2691 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 18:36:52.500723 2691 catalog_manager.cc:1314] IPKI private key was stored in cleartext, attempting to encrypt it.
I20260806 18:36:52.505657 2691 catalog_manager.cc:1324] IPKI private key successfully encrypted.
I20260806 18:36:52.510782 2691 catalog_manager.cc:1504] Loading token signing keys...
I20260806 18:36:52.511540 2691 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 422f52e3e25c47249416f4d5b21c77be: Loaded TSK: 0
I20260806 18:36:52.512661 2691 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260806 18:36:52.568604 2685 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 18:36:52.568864 2685 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 18:36:52.568943 2685 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 18:36:52.572875 2685 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 18:36:52.572949 2685 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 18:36:52.573032 2685 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.19.194.1
I20260806 18:36:52.578042 2685 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.194.20:38847
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786041246094603-20232-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786041246094603-20232-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786041246094603-20232-0/krb5kdc/kudu_127.19.194.1.keytab
--principal=kudu/127.19.194.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.19.194.1:42955
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786041246094603-20232-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.19.194.1
--webserver_port=38357
--tserver_master_addrs=127.19.194.62:40989
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2685
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.19.194.1
--log_dir=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786041246094603-20232-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 17:59:13 UTC on ccycloud.quasar-wvsnzx.root.comops.site
build id 4571028
I20260806 18:36:52.579017 2685 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 18:36:52.580061 2685 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 18:36:52.590227 2713 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:36:52.590269 2712 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 18:36:52.590479 2685 server_base.cc:1034] running on GCE node
W20260806 18:36:52.590243 2715 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 18:36:52.591262 2685 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 18:36:52.591903 2685 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 18:36:52.593096 2685 hybrid_clock.cc:648] HybridClock initialized: now 1786041412593070 us; error 41 us; skew 500 ppm
Aug 06 18:36:52 dist-test-slave-1xrh krb5kdc[2033](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041412, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.194.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 18:36:52.596093 2685 init.cc:377] Logged in from keytab as kudu/127.19.194.1@KRBTEST.COM (short username kudu)
I20260806 18:36:52.596990 2685 webserver.cc:460] Webserver started at http://127.19.194.1:38357/ using document root <none> and password file <none>
I20260806 18:36:52.597520 2685 fs_manager.cc:359] Metadata directory not provided
I20260806 18:36:52.597591 2685 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 18:36:52.601608 2685 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.003s sys 0.001s
I20260806 18:36:52.603670 2722 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 18:36:52.604925 2685 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260806 18:36:52.605026 2685 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786041246094603-20232-0/minicluster-data/ts-0/data,/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786041246094603-20232-0/minicluster-data/ts-0/wal
uuid: "e5ac26f8baab47dd8096af286ff02861"
format_stamp: "Formatted at 2026-08-06 18:36:51 on dist-test-slave-1xrh"
server_key: "ecfbc98d4b6d20352ff87523e6e75ace"
server_key_iv: "4d26e6cb9a1633f9b7144c6be6bdbdc2"
server_key_version: "encryptionkey@0"
I20260806 18:36:52.605576 2685 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786041246094603-20232-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786041246094603-20232-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786041246094603-20232-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 18:36:52.654292 2685 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 18:36:52.657661 2685 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 18:36:52.657904 2685 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 18:36:52.658524 2685 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 18:36:52.659677 2729 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260806 18:36:52.663707 2685 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260806 18:36:52.663759 2685 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.004s user 0.000s sys 0.000s
I20260806 18:36:52.663852 2685 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260806 18:36:52.665449 2685 ts_tablet_manager.cc:610] Registered 1 tablets
I20260806 18:36:52.665500 2685 ts_tablet_manager.cc:589] Time spent register tablets: real 0.002s user 0.000s sys 0.000s
I20260806 18:36:52.665759 2729 tablet_bootstrap.cc:492] T 5983348849dd46c9b7b30bda1044aa9d P e5ac26f8baab47dd8096af286ff02861: Bootstrap starting.
I20260806 18:36:52.676189 2685 rpc_server.cc:307] RPC server started. Bound to: 127.19.194.1:42955
I20260806 18:36:52.676267 2843 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.194.1:42955 every 8 connection(s)
I20260806 18:36:52.677095 2685 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786041246094603-20232-0/minicluster-data/ts-0/data/info.pb
I20260806 18:36:52.681481 20232 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJW69tn/build/debug/bin/kudu as pid 2685
I20260806 18:36:52.682024 20232 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJW69tn/build/debug/bin/kudu
/tmp/dist-test-taskJW69tn/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786041246094603-20232-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786041246094603-20232-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786041246094603-20232-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786041246094603-20232-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.19.194.2:40079
--local_ip_for_outbound_sockets=127.19.194.2
--tserver_master_addrs=127.19.194.62:40989
--webserver_port=42471
--webserver_interface=127.19.194.2
--keytab_file=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786041246094603-20232-0/krb5kdc/kudu_127.19.194.2.keytab
--principal=kudu/127.19.194.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.19.194.20:38847
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786041246094603-20232-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786041246094603-20232-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786041246094603-20232-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260806 18:36:52.684608 2729 log.cc:824] T 5983348849dd46c9b7b30bda1044aa9d P e5ac26f8baab47dd8096af286ff02861: Log is configured to *not* fsync() on all Append() calls
Aug 06 18:36:52 dist-test-slave-1xrh krb5kdc[2033](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041412, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.194.1@KRBTEST.COM for kudu/127.19.194.62@KRBTEST.COM
I20260806 18:36:52.694599 2729 tablet_bootstrap.cc:492] T 5983348849dd46c9b7b30bda1044aa9d P e5ac26f8baab47dd8096af286ff02861: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260806 18:36:52.694644 2846 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:36:52.678965 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.194.62:40989 (local address 127.19.194.1:44231)
0806 18:36:52.679655 (+ 690us) negotiation.cc:107] Waiting for socket to connect
0806 18:36:52.679708 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0806 18:36:52.680503 (+ 795us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:36:52.681950 (+ 1447us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:36:52.681962 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:36:52.682612 (+ 650us) client_negotiation.cc:189] Negotiated authn=SASL
0806 18:36:52.683309 (+ 697us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:36:52.683334 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:36:52.685196 (+ 1862us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:36:52.685203 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:36:52.686197 (+ 994us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:36:52.686208 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:36:52.687025 (+ 817us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:36:52.687032 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:36:52.687253 (+ 221us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:36:52.687864 (+ 611us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 18:36:52.687886 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0806 18:36:52.689318 (+ 1432us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 18:36:52.691557 (+ 2239us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:36:52.691563 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:36:52.691574 (+ 11us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:36:52.691825 (+ 251us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:36:52.692173 (+ 348us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:36:52.692177 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:36:52.692179 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:36:52.692315 (+ 136us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:36:52.692959 (+ 644us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 18:36:52.692964 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 18:36:52.693194 (+ 230us) client_negotiation.cc:769] Sending connection context
0806 18:36:52.693368 (+ 174us) client_negotiation.cc:240] Negotiation successful
0806 18:36:52.693764 (+ 396us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":486,"thread_start_us":192,"threads_started":1}
I20260806 18:36:52.694633 2847 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:36:52.679091 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.194.1:44231 (local address 127.19.194.62:40989)
0806 18:36:52.679562 (+ 471us) server_negotiation.cc:206] Beginning negotiation
0806 18:36:52.679586 (+ 24us) server_negotiation.cc:399] Waiting for connection header
0806 18:36:52.679839 (+ 253us) server_negotiation.cc:407] Connection header received
0806 18:36:52.680977 (+ 1138us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:36:52.680998 (+ 21us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:36:52.681469 (+ 471us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:36:52.681820 (+ 351us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:36:52.683510 (+ 1690us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:36:52.685023 (+ 1513us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:36:52.686339 (+ 1316us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:36:52.686878 (+ 539us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:36:52.687066 (+ 188us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:36:52.689517 (+ 2451us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:36:52.689522 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:36:52.689533 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 18:36:52.689560 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:36:52.691436 (+ 1876us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:36:52.691960 (+ 524us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:36:52.691965 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:36:52.691969 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:36:52.692077 (+ 108us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:36:52.692444 (+ 367us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:36:52.692452 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:36:52.692454 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:36:52.692836 (+ 382us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:36:52.692987 (+ 151us) server_negotiation.cc:1035] Waiting for connection context
0806 18:36:52.693545 (+ 558us) server_negotiation.cc:299] Negotiation successful
0806 18:36:52.693847 (+ 302us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":284,"thread_start_us":117,"threads_started":1}
I20260806 18:36:52.695158 2729 tablet_bootstrap.cc:492] T 5983348849dd46c9b7b30bda1044aa9d P e5ac26f8baab47dd8096af286ff02861: Bootstrap complete.
I20260806 18:36:52.696692 2729 ts_tablet_manager.cc:1397] T 5983348849dd46c9b7b30bda1044aa9d P e5ac26f8baab47dd8096af286ff02861: Time spent bootstrapping tablet: real 0.031s user 0.017s sys 0.012s
I20260806 18:36:52.697288 2844 heartbeater.cc:347] Connected to a master server at 127.19.194.62:40989
I20260806 18:36:52.697532 2844 heartbeater.cc:464] Registering TS with master...
I20260806 18:36:52.697996 2844 heartbeater.cc:511] Master 127.19.194.62:40989 requested a full tablet report, sending...
I20260806 18:36:52.699528 2729 raft_consensus.cc:348] T 5983348849dd46c9b7b30bda1044aa9d P e5ac26f8baab47dd8096af286ff02861 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f52680fa3a54bd2983fe38ae400b5d5" member_type: VOTER last_known_addr { host: "127.19.194.3" port: 41377 } } peers { permanent_uuid: "e5ac26f8baab47dd8096af286ff02861" member_type: VOTER last_known_addr { host: "127.19.194.1" port: 42955 } } peers { permanent_uuid: "61d7bb0f87d04b1b85aa1dbf30e6bc74" member_type: VOTER last_known_addr { host: "127.19.194.2" port: 40079 } }
I20260806 18:36:52.700023 2729 raft_consensus.cc:729] T 5983348849dd46c9b7b30bda1044aa9d P e5ac26f8baab47dd8096af286ff02861 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: e5ac26f8baab47dd8096af286ff02861, State: Initialized, Role: FOLLOWER
I20260806 18:36:52.700186 2645 ts_manager.cc:194] Registered new tserver with Master: e5ac26f8baab47dd8096af286ff02861 (127.19.194.1:42955)
I20260806 18:36:52.700476 2729 consensus_queue.cc:260] T 5983348849dd46c9b7b30bda1044aa9d P e5ac26f8baab47dd8096af286ff02861 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f52680fa3a54bd2983fe38ae400b5d5" member_type: VOTER last_known_addr { host: "127.19.194.3" port: 41377 } } peers { permanent_uuid: "e5ac26f8baab47dd8096af286ff02861" member_type: VOTER last_known_addr { host: "127.19.194.1" port: 42955 } } peers { permanent_uuid: "61d7bb0f87d04b1b85aa1dbf30e6bc74" member_type: VOTER last_known_addr { host: "127.19.194.2" port: 40079 } }
I20260806 18:36:52.701481 2729 ts_tablet_manager.cc:1428] T 5983348849dd46c9b7b30bda1044aa9d P e5ac26f8baab47dd8096af286ff02861: Time spent starting tablet: real 0.005s user 0.006s sys 0.000s
I20260806 18:36:52.704128 2645 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.19.194.1@KRBTEST.COM'} at 127.19.194.1:44231
I20260806 18:36:52.706422 2844 heartbeater.cc:503] Master 127.19.194.62:40989 was elected leader, sending a full tablet report...
I20260806 18:36:52.711463 2854 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:36:52.704394 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43048 (local address 127.19.194.1:42955)
0806 18:36:52.704745 (+ 351us) server_negotiation.cc:206] Beginning negotiation
0806 18:36:52.704758 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0806 18:36:52.706744 (+ 1986us) server_negotiation.cc:407] Connection header received
0806 18:36:52.706796 (+ 52us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:36:52.706800 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:36:52.707025 (+ 225us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:36:52.707206 (+ 181us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 18:36:52.707532 (+ 326us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:36:52.708448 (+ 916us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:36:52.710128 (+ 1680us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:36:52.710840 (+ 712us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:36:52.710928 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:36:52.711015 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0806 18:36:52.711270 (+ 255us) server_negotiation.cc:299] Negotiation successful
0806 18:36:52.711340 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":265,"thread_start_us":95,"threads_started":1}
I20260806 18:36:52.711525 2853 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:36:52.703819 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.194.1:42955 (local address 127.0.0.1:43048)
0806 18:36:52.705099 (+ 1280us) negotiation.cc:107] Waiting for socket to connect
0806 18:36:52.705130 (+ 31us) client_negotiation.cc:174] Beginning negotiation
0806 18:36:52.705272 (+ 142us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:36:52.707218 (+ 1946us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:36:52.707225 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:36:52.707267 (+ 42us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 18:36:52.707404 (+ 137us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:36:52.707413 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:36:52.708567 (+ 1154us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:36:52.708573 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:36:52.709983 (+ 1410us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:36:52.709995 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:36:52.710955 (+ 960us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:36:52.710959 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:36:52.711069 (+ 110us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:36:52.711107 (+ 38us) client_negotiation.cc:769] Sending connection context
0806 18:36:52.711240 (+ 133us) client_negotiation.cc:240] Negotiation successful
0806 18:36:52.711371 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1116,"thread_start_us":143,"threads_started":1}
I20260806 18:36:52.712463 2773 tablet_service.cc:1514] Processing DeleteTablet for tablet 5983348849dd46c9b7b30bda1044aa9d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 18:36:52 UTC) from {username='kudu', principal='kudu/127.19.194.62@KRBTEST.COM'} at 127.0.0.1:43048
I20260806 18:36:52.713238 2855 tablet_replica.cc:330] stopping tablet replica
I20260806 18:36:52.713517 2855 raft_consensus.cc:2229] T 5983348849dd46c9b7b30bda1044aa9d P e5ac26f8baab47dd8096af286ff02861 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 18:36:52.713739 2855 raft_consensus.cc:2258] T 5983348849dd46c9b7b30bda1044aa9d P e5ac26f8baab47dd8096af286ff02861 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 18:36:52.714609 2855 ts_tablet_manager.cc:1905] T 5983348849dd46c9b7b30bda1044aa9d P e5ac26f8baab47dd8096af286ff02861: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 18:36:52.717937 2855 ts_tablet_manager.cc:1918] T 5983348849dd46c9b7b30bda1044aa9d P e5ac26f8baab47dd8096af286ff02861: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 18:36:52.718017 2855 log.cc:1196] T 5983348849dd46c9b7b30bda1044aa9d P e5ac26f8baab47dd8096af286ff02861: Deleting WAL directory at /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786041246094603-20232-0/minicluster-data/ts-0/wal/wals/5983348849dd46c9b7b30bda1044aa9d
I20260806 18:36:52.718317 2855 ts_tablet_manager.cc:1939] T 5983348849dd46c9b7b30bda1044aa9d P e5ac26f8baab47dd8096af286ff02861: Deleting consensus metadata
I20260806 18:36:52.719847 2626 catalog_manager.cc:4841] TS e5ac26f8baab47dd8096af286ff02861 (127.19.194.1:42955): tablet 5983348849dd46c9b7b30bda1044aa9d (table test-table [id=386534c1b94948cd97b99a6eed781718]) successfully deleted
W20260806 18:36:52.800480 2850 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 18:36:52.800738 2850 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 18:36:52.800806 2850 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 18:36:52.804682 2850 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 18:36:52.804759 2850 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 18:36:52.804852 2850 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.19.194.2
I20260806 18:36:52.810029 2850 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.194.20:38847
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786041246094603-20232-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786041246094603-20232-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786041246094603-20232-0/krb5kdc/kudu_127.19.194.2.keytab
--principal=kudu/127.19.194.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.19.194.2:40079
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786041246094603-20232-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.19.194.2
--webserver_port=42471
--tserver_master_addrs=127.19.194.62:40989
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2850
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.19.194.2
--log_dir=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786041246094603-20232-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 17:59:13 UTC on ccycloud.quasar-wvsnzx.root.comops.site
build id 4571028
I20260806 18:36:52.810993 2850 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 18:36:52.811987 2850 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 18:36:52.821219 2862 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 18:36:52.821254 2850 server_base.cc:1034] running on GCE node
W20260806 18:36:52.821159 2861 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:36:52.821172 2864 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 18:36:52.821801 2850 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 18:36:52.822476 2850 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 18:36:52.823693 2850 hybrid_clock.cc:648] HybridClock initialized: now 1786041412823683 us; error 29 us; skew 500 ppm
Aug 06 18:36:52 dist-test-slave-1xrh krb5kdc[2033](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041412, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.194.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 18:36:52.826337 2850 init.cc:377] Logged in from keytab as kudu/127.19.194.2@KRBTEST.COM (short username kudu)
I20260806 18:36:52.827198 2850 webserver.cc:460] Webserver started at http://127.19.194.2:42471/ using document root <none> and password file <none>
I20260806 18:36:52.827736 2850 fs_manager.cc:359] Metadata directory not provided
I20260806 18:36:52.827847 2850 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 18:36:52.831600 2850 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.004s sys 0.000s
I20260806 18:36:52.833593 2871 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 18:36:52.834702 2850 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260806 18:36:52.834833 2850 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786041246094603-20232-0/minicluster-data/ts-1/data,/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786041246094603-20232-0/minicluster-data/ts-1/wal
uuid: "61d7bb0f87d04b1b85aa1dbf30e6bc74"
format_stamp: "Formatted at 2026-08-06 18:36:51 on dist-test-slave-1xrh"
server_key: "3aae3b8c41c1da2cab58e82a12f2c908"
server_key_iv: "caba3493fde26304f548f1862222dadb"
server_key_version: "encryptionkey@0"
I20260806 18:36:52.835419 2850 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786041246094603-20232-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786041246094603-20232-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786041246094603-20232-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 18:36:52.860060 2850 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 18:36:52.862723 2850 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 18:36:52.862920 2850 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 18:36:52.863490 2850 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 18:36:52.864269 2850 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 18:36:52.864351 2850 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 18:36:52.864424 2850 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 18:36:52.864477 2850 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 18:36:52.873401 2850 rpc_server.cc:307] RPC server started. Bound to: 127.19.194.2:40079
I20260806 18:36:52.873418 2995 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.194.2:40079 every 8 connection(s)
I20260806 18:36:52.874294 2850 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786041246094603-20232-0/minicluster-data/ts-1/data/info.pb
I20260806 18:36:52.879631 20232 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJW69tn/build/debug/bin/kudu as pid 2850
I20260806 18:36:52.880060 20232 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJW69tn/build/debug/bin/kudu
/tmp/dist-test-taskJW69tn/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786041246094603-20232-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786041246094603-20232-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786041246094603-20232-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786041246094603-20232-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.19.194.3:41377
--local_ip_for_outbound_sockets=127.19.194.3
--tserver_master_addrs=127.19.194.62:40989
--webserver_port=43387
--webserver_interface=127.19.194.3
--keytab_file=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786041246094603-20232-0/krb5kdc/kudu_127.19.194.3.keytab
--principal=kudu/127.19.194.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.19.194.20:38847
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786041246094603-20232-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786041246094603-20232-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786041246094603-20232-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 06 18:36:52 dist-test-slave-1xrh krb5kdc[2033](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041412, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.194.2@KRBTEST.COM for kudu/127.19.194.62@KRBTEST.COM
I20260806 18:36:52.888105 2847 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:36:52.876037 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.194.2:33017 (local address 127.19.194.62:40989)
0806 18:36:52.876172 (+ 135us) server_negotiation.cc:206] Beginning negotiation
0806 18:36:52.876176 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 18:36:52.876963 (+ 787us) server_negotiation.cc:407] Connection header received
0806 18:36:52.878233 (+ 1270us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:36:52.878237 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:36:52.878285 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:36:52.878356 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:36:52.879363 (+ 1007us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:36:52.879962 (+ 599us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:36:52.881686 (+ 1724us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:36:52.882114 (+ 428us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:36:52.882232 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:36:52.884578 (+ 2346us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:36:52.884584 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:36:52.884591 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 18:36:52.884615 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:36:52.886079 (+ 1464us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:36:52.886584 (+ 505us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:36:52.886588 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:36:52.886590 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:36:52.886680 (+ 90us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:36:52.887006 (+ 326us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:36:52.887010 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:36:52.887012 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:36:52.887252 (+ 240us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:36:52.887358 (+ 106us) server_negotiation.cc:1035] Waiting for connection context
0806 18:36:52.887806 (+ 448us) server_negotiation.cc:299] Negotiation successful
0806 18:36:52.887941 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260806 18:36:52.888780 2998 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:36:52.876376 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.194.62:40989 (local address 127.19.194.2:33017)
0806 18:36:52.876803 (+ 427us) negotiation.cc:107] Waiting for socket to connect
0806 18:36:52.876849 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0806 18:36:52.878045 (+ 1196us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:36:52.878480 (+ 435us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:36:52.878485 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:36:52.878793 (+ 308us) client_negotiation.cc:189] Negotiated authn=SASL
0806 18:36:52.879238 (+ 445us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:36:52.879252 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:36:52.880098 (+ 846us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:36:52.880102 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:36:52.881536 (+ 1434us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:36:52.881551 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:36:52.882264 (+ 713us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:36:52.882267 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:36:52.882400 (+ 133us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:36:52.882944 (+ 544us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 18:36:52.882960 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0806 18:36:52.884410 (+ 1450us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 18:36:52.886199 (+ 1789us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:36:52.886205 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:36:52.886218 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:36:52.886475 (+ 257us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:36:52.886771 (+ 296us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:36:52.886774 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:36:52.886776 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:36:52.886909 (+ 133us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:36:52.887369 (+ 460us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 18:36:52.887376 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 18:36:52.887588 (+ 212us) client_negotiation.cc:769] Sending connection context
0806 18:36:52.887761 (+ 173us) client_negotiation.cc:240] Negotiation successful
0806 18:36:52.888043 (+ 282us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":259,"thread_start_us":122,"threads_started":1}
I20260806 18:36:52.890005 2996 heartbeater.cc:347] Connected to a master server at 127.19.194.62:40989
I20260806 18:36:52.890244 2996 heartbeater.cc:464] Registering TS with master...
I20260806 18:36:52.890727 2996 heartbeater.cc:511] Master 127.19.194.62:40989 requested a full tablet report, sending...
I20260806 18:36:52.891788 2645 ts_manager.cc:194] Registered new tserver with Master: 61d7bb0f87d04b1b85aa1dbf30e6bc74 (127.19.194.2:40079)
I20260806 18:36:52.892354 2645 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.19.194.2@KRBTEST.COM'} at 127.19.194.2:33017
W20260806 18:36:52.991959 2999 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 18:36:52.992237 2999 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 18:36:52.992321 2999 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 18:36:52.996774 2999 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 18:36:52.996856 2999 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 18:36:52.996946 2999 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.19.194.3
I20260806 18:36:53.002280 2999 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.194.20:38847
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786041246094603-20232-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786041246094603-20232-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786041246094603-20232-0/krb5kdc/kudu_127.19.194.3.keytab
--principal=kudu/127.19.194.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.19.194.3:41377
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786041246094603-20232-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.19.194.3
--webserver_port=43387
--tserver_master_addrs=127.19.194.62:40989
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2999
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.19.194.3
--log_dir=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786041246094603-20232-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 17:59:13 UTC on ccycloud.quasar-wvsnzx.root.comops.site
build id 4571028
I20260806 18:36:53.003319 2999 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 18:36:53.004311 2999 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 18:36:53.013896 3004 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:36:53.013900 3005 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 18:36:53.014153 2999 server_base.cc:1034] running on GCE node
W20260806 18:36:53.013959 3007 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 18:36:53.014549 2999 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 18:36:53.015286 2999 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 18:36:53.016533 2999 hybrid_clock.cc:648] HybridClock initialized: now 1786041413016501 us; error 30 us; skew 500 ppm
Aug 06 18:36:53 dist-test-slave-1xrh krb5kdc[2033](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041413, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.194.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 18:36:53.019219 2999 init.cc:377] Logged in from keytab as kudu/127.19.194.3@KRBTEST.COM (short username kudu)
I20260806 18:36:53.020081 2999 webserver.cc:460] Webserver started at http://127.19.194.3:43387/ using document root <none> and password file <none>
I20260806 18:36:53.020620 2999 fs_manager.cc:359] Metadata directory not provided
I20260806 18:36:53.020726 2999 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 18:36:53.024464 2999 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.004s sys 0.000s
I20260806 18:36:53.026363 3014 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 18:36:53.027648 2999 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260806 18:36:53.027787 2999 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786041246094603-20232-0/minicluster-data/ts-2/data,/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786041246094603-20232-0/minicluster-data/ts-2/wal
uuid: "1f52680fa3a54bd2983fe38ae400b5d5"
format_stamp: "Formatted at 2026-08-06 18:36:51 on dist-test-slave-1xrh"
server_key: "21c572fb9bd202689166d5ee43308cb2"
server_key_iv: "4916fbff0e77d6686fd222feaf1e5332"
server_key_version: "encryptionkey@0"
I20260806 18:36:53.028551 2999 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786041246094603-20232-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786041246094603-20232-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786041246094603-20232-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 18:36:53.041855 2999 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 18:36:53.044632 2999 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 18:36:53.044821 2999 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 18:36:53.045356 2999 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 18:36:53.046116 2999 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 18:36:53.046185 2999 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 18:36:53.046254 2999 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 18:36:53.046307 2999 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 18:36:53.055157 3134 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.194.3:41377 every 8 connection(s)
I20260806 18:36:53.055154 2999 rpc_server.cc:307] RPC server started. Bound to: 127.19.194.3:41377
I20260806 18:36:53.056121 2999 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786041246094603-20232-0/minicluster-data/ts-2/data/info.pb
I20260806 18:36:53.057145 20232 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJW69tn/build/debug/bin/kudu as pid 2999
Aug 06 18:36:53 dist-test-slave-1xrh krb5kdc[2033](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041413, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.194.3@KRBTEST.COM for kudu/127.19.194.62@KRBTEST.COM
I20260806 18:36:53.066654 2847 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:36:53.057851 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59916 (local address 127.19.194.62:40989)
0806 18:36:53.059250 (+ 1399us) server_negotiation.cc:206] Beginning negotiation
0806 18:36:53.059255 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 18:36:53.059271 (+ 16us) server_negotiation.cc:407] Connection header received
0806 18:36:53.059318 (+ 47us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:36:53.059321 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:36:53.059380 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:36:53.059477 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:36:53.060258 (+ 781us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:36:53.060765 (+ 507us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:36:53.061684 (+ 919us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:36:53.061908 (+ 224us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:36:53.062026 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:36:53.063478 (+ 1452us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:36:53.063482 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:36:53.063484 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 18:36:53.063514 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:36:53.064923 (+ 1409us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:36:53.065411 (+ 488us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:36:53.065415 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:36:53.065416 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:36:53.065520 (+ 104us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:36:53.065821 (+ 301us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:36:53.065824 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:36:53.065826 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:36:53.065997 (+ 171us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:36:53.066213 (+ 216us) server_negotiation.cc:1035] Waiting for connection context
0806 18:36:53.066351 (+ 138us) server_negotiation.cc:299] Negotiation successful
0806 18:36:53.066466 (+ 115us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1319}
I20260806 18:36:53.069622 3140 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:36:53.058997 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.194.3:51577 (local address 127.19.194.62:40989)
0806 18:36:53.059250 (+ 253us) server_negotiation.cc:206] Beginning negotiation
0806 18:36:53.059255 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 18:36:53.059271 (+ 16us) server_negotiation.cc:407] Connection header received
0806 18:36:53.060151 (+ 880us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:36:53.060155 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:36:53.060211 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:36:53.060303 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:36:53.061386 (+ 1083us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:36:53.061899 (+ 513us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:36:53.063186 (+ 1287us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:36:53.063425 (+ 239us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:36:53.063533 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:36:53.065991 (+ 2458us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:36:53.065996 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:36:53.066003 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 18:36:53.066025 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:36:53.067752 (+ 1727us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:36:53.068292 (+ 540us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:36:53.068294 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:36:53.068295 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:36:53.068400 (+ 105us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:36:53.068762 (+ 362us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:36:53.068764 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:36:53.068765 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:36:53.068916 (+ 151us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:36:53.068990 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0806 18:36:53.069424 (+ 434us) server_negotiation.cc:299] Negotiation successful
0806 18:36:53.069515 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":201,"thread_start_us":97,"threads_started":1}
I20260806 18:36:53.070363 3139 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:36:53.058279 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.194.62:40989 (local address 127.19.194.3:51577)
0806 18:36:53.058706 (+ 427us) negotiation.cc:107] Waiting for socket to connect
0806 18:36:53.058744 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0806 18:36:53.059930 (+ 1186us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:36:53.060476 (+ 546us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:36:53.060482 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:36:53.060783 (+ 301us) client_negotiation.cc:189] Negotiated authn=SASL
0806 18:36:53.061221 (+ 438us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:36:53.061233 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:36:53.062039 (+ 806us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:36:53.062043 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:36:53.063032 (+ 989us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:36:53.063042 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:36:53.063572 (+ 530us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:36:53.063576 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:36:53.063797 (+ 221us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:36:53.064374 (+ 577us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 18:36:53.064397 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0806 18:36:53.065836 (+ 1439us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 18:36:53.067860 (+ 2024us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:36:53.067866 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:36:53.067878 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:36:53.068192 (+ 314us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:36:53.068499 (+ 307us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:36:53.068502 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:36:53.068503 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:36:53.068672 (+ 169us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:36:53.069007 (+ 335us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 18:36:53.069011 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 18:36:53.069225 (+ 214us) client_negotiation.cc:769] Sending connection context
0806 18:36:53.069375 (+ 150us) client_negotiation.cc:240] Negotiation successful
0806 18:36:53.069593 (+ 218us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":260,"thread_start_us":102,"threads_started":1}
I20260806 18:36:53.071475 3136 heartbeater.cc:347] Connected to a master server at 127.19.194.62:40989
I20260806 18:36:53.071700 3136 heartbeater.cc:464] Registering TS with master...
I20260806 18:36:53.072147 3136 heartbeater.cc:511] Master 127.19.194.62:40989 requested a full tablet report, sending...
I20260806 18:36:53.073109 2645 ts_manager.cc:194] Registered new tserver with Master: 1f52680fa3a54bd2983fe38ae400b5d5 (127.19.194.3:41377)
I20260806 18:36:53.073637 2645 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.19.194.3@KRBTEST.COM'} at 127.19.194.3:51577
I20260806 18:36:53.080363 20232 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260806 18:36:53.084734 3140 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:36:53.080931 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59920 (local address 127.19.194.62:40989)
0806 18:36:53.081088 (+ 157us) server_negotiation.cc:206] Beginning negotiation
0806 18:36:53.081093 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 18:36:53.081289 (+ 196us) server_negotiation.cc:407] Connection header received
0806 18:36:53.081463 (+ 174us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:36:53.081466 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:36:53.081573 (+ 107us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:36:53.081675 (+ 102us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 18:36:53.081937 (+ 262us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:36:53.082425 (+ 488us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:36:53.083349 (+ 924us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:36:53.083571 (+ 222us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:36:53.083680 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:36:53.084011 (+ 331us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 18:36:53.084335 (+ 324us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 18:36:53.084464 (+ 129us) server_negotiation.cc:1035] Waiting for connection context
0806 18:36:53.084557 (+ 93us) server_negotiation.cc:299] Negotiation successful
0806 18:36:53.084612 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260806 18:36:53.085300 2645 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin'} at 127.0.0.1:59920:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260806 18:36:53.088418 2645 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 18:36:53.096349 2773 tablet_service.cc:1467] Processing CreateTablet for tablet 3240790c26d44607b59edcda89b45355 (DEFAULT_TABLE table=test-table [id=13e6cbf5e7ed42c9bc69da63a11a517d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 18:36:53.097012 2773 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3240790c26d44607b59edcda89b45355. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 18:36:53.099854 2729 tablet_bootstrap.cc:492] T 3240790c26d44607b59edcda89b45355 P e5ac26f8baab47dd8096af286ff02861: Bootstrap starting.
I20260806 18:36:53.100260 2853 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:36:53.095025 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.194.2:40079 (local address 127.0.0.1:50866)
0806 18:36:53.095189 (+ 164us) negotiation.cc:107] Waiting for socket to connect
0806 18:36:53.095203 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0806 18:36:53.095279 (+ 76us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:36:53.096299 (+ 1020us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:36:53.096302 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:36:53.096318 (+ 16us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 18:36:53.096423 (+ 105us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:36:53.096433 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:36:53.097721 (+ 1288us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:36:53.097724 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:36:53.098758 (+ 1034us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:36:53.098768 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:36:53.099869 (+ 1101us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:36:53.099873 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:36:53.099964 (+ 91us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:36:53.099992 (+ 28us) client_negotiation.cc:769] Sending connection context
0806 18:36:53.100048 (+ 56us) client_negotiation.cc:240] Negotiation successful
0806 18:36:53.100130 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":57,"mutex_wait_us":31}
I20260806 18:36:53.100835 3145 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:36:53.095350 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.194.3:41377 (local address 127.0.0.1:47610)
0806 18:36:53.095893 (+ 543us) negotiation.cc:107] Waiting for socket to connect
0806 18:36:53.095907 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0806 18:36:53.096004 (+ 97us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:36:53.096593 (+ 589us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:36:53.096596 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:36:53.096612 (+ 16us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 18:36:53.096694 (+ 82us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:36:53.096699 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:36:53.097871 (+ 1172us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:36:53.097874 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:36:53.099004 (+ 1130us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:36:53.099016 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:36:53.100371 (+ 1355us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:36:53.100375 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:36:53.100488 (+ 113us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:36:53.100509 (+ 21us) client_negotiation.cc:769] Sending connection context
0806 18:36:53.100579 (+ 70us) client_negotiation.cc:240] Negotiation successful
0806 18:36:53.100646 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":471,"mutex_wait_us":1,"thread_start_us":121,"threads_started":1}
I20260806 18:36:53.100837 2729 tablet_bootstrap.cc:654] T 3240790c26d44607b59edcda89b45355 P e5ac26f8baab47dd8096af286ff02861: Neither blocks nor log segments found. Creating new log.
I20260806 18:36:53.101106 3146 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:36:53.095469 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50866 (local address 127.19.194.2:40079)
0806 18:36:53.095825 (+ 356us) server_negotiation.cc:206] Beginning negotiation
0806 18:36:53.095837 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0806 18:36:53.095861 (+ 24us) server_negotiation.cc:407] Connection header received
0806 18:36:53.095938 (+ 77us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:36:53.095947 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:36:53.096152 (+ 205us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:36:53.096326 (+ 174us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 18:36:53.096510 (+ 184us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:36:53.097569 (+ 1059us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:36:53.098894 (+ 1325us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:36:53.099717 (+ 823us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:36:53.100456 (+ 739us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:36:53.100783 (+ 327us) server_negotiation.cc:1035] Waiting for connection context
0806 18:36:53.100850 (+ 67us) server_negotiation.cc:299] Negotiation successful
0806 18:36:53.100960 (+ 110us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":241,"thread_start_us":119,"threads_started":1}
I20260806 18:36:53.101284 3147 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:36:53.095490 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47610 (local address 127.19.194.3:41377)
0806 18:36:53.095906 (+ 416us) server_negotiation.cc:206] Beginning negotiation
0806 18:36:53.095920 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0806 18:36:53.095975 (+ 55us) server_negotiation.cc:407] Connection header received
0806 18:36:53.096144 (+ 169us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:36:53.096150 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:36:53.096352 (+ 202us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:36:53.096505 (+ 153us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 18:36:53.096905 (+ 400us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:36:53.097723 (+ 818us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:36:53.099178 (+ 1455us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:36:53.100072 (+ 894us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:36:53.100175 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:36:53.100306 (+ 131us) server_negotiation.cc:1035] Waiting for connection context
0806 18:36:53.100986 (+ 680us) server_negotiation.cc:299] Negotiation successful
0806 18:36:53.101074 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":331,"thread_start_us":164,"threads_started":1}
I20260806 18:36:53.102802 2729 tablet_bootstrap.cc:492] T 3240790c26d44607b59edcda89b45355 P e5ac26f8baab47dd8096af286ff02861: No bootstrap required, opened a new log
I20260806 18:36:53.102901 2729 ts_tablet_manager.cc:1397] T 3240790c26d44607b59edcda89b45355 P e5ac26f8baab47dd8096af286ff02861: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.000s
I20260806 18:36:53.103546 2729 raft_consensus.cc:348] T 3240790c26d44607b59edcda89b45355 P e5ac26f8baab47dd8096af286ff02861 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "61d7bb0f87d04b1b85aa1dbf30e6bc74" member_type: VOTER last_known_addr { host: "127.19.194.2" port: 40079 } } peers { permanent_uuid: "1f52680fa3a54bd2983fe38ae400b5d5" member_type: VOTER last_known_addr { host: "127.19.194.3" port: 41377 } } peers { permanent_uuid: "e5ac26f8baab47dd8096af286ff02861" member_type: VOTER last_known_addr { host: "127.19.194.1" port: 42955 } }
I20260806 18:36:53.103672 2729 raft_consensus.cc:374] T 3240790c26d44607b59edcda89b45355 P e5ac26f8baab47dd8096af286ff02861 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 18:36:53.103711 2729 raft_consensus.cc:729] T 3240790c26d44607b59edcda89b45355 P e5ac26f8baab47dd8096af286ff02861 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e5ac26f8baab47dd8096af286ff02861, State: Initialized, Role: FOLLOWER
I20260806 18:36:53.103863 2729 consensus_queue.cc:260] T 3240790c26d44607b59edcda89b45355 P e5ac26f8baab47dd8096af286ff02861 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "61d7bb0f87d04b1b85aa1dbf30e6bc74" member_type: VOTER last_known_addr { host: "127.19.194.2" port: 40079 } } peers { permanent_uuid: "1f52680fa3a54bd2983fe38ae400b5d5" member_type: VOTER last_known_addr { host: "127.19.194.3" port: 41377 } } peers { permanent_uuid: "e5ac26f8baab47dd8096af286ff02861" member_type: VOTER last_known_addr { host: "127.19.194.1" port: 42955 } }
I20260806 18:36:53.104135 2729 ts_tablet_manager.cc:1428] T 3240790c26d44607b59edcda89b45355 P e5ac26f8baab47dd8096af286ff02861: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260806 18:36:53.104074 3066 tablet_service.cc:1467] Processing CreateTablet for tablet 3240790c26d44607b59edcda89b45355 (DEFAULT_TABLE table=test-table [id=13e6cbf5e7ed42c9bc69da63a11a517d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 18:36:53.104089 2922 tablet_service.cc:1467] Processing CreateTablet for tablet 3240790c26d44607b59edcda89b45355 (DEFAULT_TABLE table=test-table [id=13e6cbf5e7ed42c9bc69da63a11a517d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 18:36:53.105135 2922 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3240790c26d44607b59edcda89b45355. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 18:36:53.105145 3066 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3240790c26d44607b59edcda89b45355. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 18:36:53.111649 3149 tablet_bootstrap.cc:492] T 3240790c26d44607b59edcda89b45355 P 1f52680fa3a54bd2983fe38ae400b5d5: Bootstrap starting.
I20260806 18:36:53.111857 3150 tablet_bootstrap.cc:492] T 3240790c26d44607b59edcda89b45355 P 61d7bb0f87d04b1b85aa1dbf30e6bc74: Bootstrap starting.
I20260806 18:36:53.113937 3149 tablet_bootstrap.cc:654] T 3240790c26d44607b59edcda89b45355 P 1f52680fa3a54bd2983fe38ae400b5d5: Neither blocks nor log segments found. Creating new log.
I20260806 18:36:53.114374 3150 tablet_bootstrap.cc:654] T 3240790c26d44607b59edcda89b45355 P 61d7bb0f87d04b1b85aa1dbf30e6bc74: Neither blocks nor log segments found. Creating new log.
I20260806 18:36:53.115032 3150 log.cc:824] T 3240790c26d44607b59edcda89b45355 P 61d7bb0f87d04b1b85aa1dbf30e6bc74: Log is configured to *not* fsync() on all Append() calls
I20260806 18:36:53.115033 3149 log.cc:824] T 3240790c26d44607b59edcda89b45355 P 1f52680fa3a54bd2983fe38ae400b5d5: Log is configured to *not* fsync() on all Append() calls
I20260806 18:36:53.116701 3149 tablet_bootstrap.cc:492] T 3240790c26d44607b59edcda89b45355 P 1f52680fa3a54bd2983fe38ae400b5d5: No bootstrap required, opened a new log
I20260806 18:36:53.116739 3150 tablet_bootstrap.cc:492] T 3240790c26d44607b59edcda89b45355 P 61d7bb0f87d04b1b85aa1dbf30e6bc74: No bootstrap required, opened a new log
I20260806 18:36:53.116899 3149 ts_tablet_manager.cc:1397] T 3240790c26d44607b59edcda89b45355 P 1f52680fa3a54bd2983fe38ae400b5d5: Time spent bootstrapping tablet: real 0.005s user 0.001s sys 0.003s
I20260806 18:36:53.116907 3150 ts_tablet_manager.cc:1397] T 3240790c26d44607b59edcda89b45355 P 61d7bb0f87d04b1b85aa1dbf30e6bc74: Time spent bootstrapping tablet: real 0.005s user 0.004s sys 0.000s
I20260806 18:36:53.120034 3150 raft_consensus.cc:348] T 3240790c26d44607b59edcda89b45355 P 61d7bb0f87d04b1b85aa1dbf30e6bc74 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "61d7bb0f87d04b1b85aa1dbf30e6bc74" member_type: VOTER last_known_addr { host: "127.19.194.2" port: 40079 } } peers { permanent_uuid: "1f52680fa3a54bd2983fe38ae400b5d5" member_type: VOTER last_known_addr { host: "127.19.194.3" port: 41377 } } peers { permanent_uuid: "e5ac26f8baab47dd8096af286ff02861" member_type: VOTER last_known_addr { host: "127.19.194.1" port: 42955 } }
I20260806 18:36:53.120042 3149 raft_consensus.cc:348] T 3240790c26d44607b59edcda89b45355 P 1f52680fa3a54bd2983fe38ae400b5d5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "61d7bb0f87d04b1b85aa1dbf30e6bc74" member_type: VOTER last_known_addr { host: "127.19.194.2" port: 40079 } } peers { permanent_uuid: "1f52680fa3a54bd2983fe38ae400b5d5" member_type: VOTER last_known_addr { host: "127.19.194.3" port: 41377 } } peers { permanent_uuid: "e5ac26f8baab47dd8096af286ff02861" member_type: VOTER last_known_addr { host: "127.19.194.1" port: 42955 } }
I20260806 18:36:53.120220 3150 raft_consensus.cc:374] T 3240790c26d44607b59edcda89b45355 P 61d7bb0f87d04b1b85aa1dbf30e6bc74 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 18:36:53.120247 3149 raft_consensus.cc:374] T 3240790c26d44607b59edcda89b45355 P 1f52680fa3a54bd2983fe38ae400b5d5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 18:36:53.120251 3150 raft_consensus.cc:729] T 3240790c26d44607b59edcda89b45355 P 61d7bb0f87d04b1b85aa1dbf30e6bc74 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 61d7bb0f87d04b1b85aa1dbf30e6bc74, State: Initialized, Role: FOLLOWER
I20260806 18:36:53.120325 3149 raft_consensus.cc:729] T 3240790c26d44607b59edcda89b45355 P 1f52680fa3a54bd2983fe38ae400b5d5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1f52680fa3a54bd2983fe38ae400b5d5, State: Initialized, Role: FOLLOWER
I20260806 18:36:53.120743 3150 consensus_queue.cc:260] T 3240790c26d44607b59edcda89b45355 P 61d7bb0f87d04b1b85aa1dbf30e6bc74 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "61d7bb0f87d04b1b85aa1dbf30e6bc74" member_type: VOTER last_known_addr { host: "127.19.194.2" port: 40079 } } peers { permanent_uuid: "1f52680fa3a54bd2983fe38ae400b5d5" member_type: VOTER last_known_addr { host: "127.19.194.3" port: 41377 } } peers { permanent_uuid: "e5ac26f8baab47dd8096af286ff02861" member_type: VOTER last_known_addr { host: "127.19.194.1" port: 42955 } }
I20260806 18:36:53.120759 3149 consensus_queue.cc:260] T 3240790c26d44607b59edcda89b45355 P 1f52680fa3a54bd2983fe38ae400b5d5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "61d7bb0f87d04b1b85aa1dbf30e6bc74" member_type: VOTER last_known_addr { host: "127.19.194.2" port: 40079 } } peers { permanent_uuid: "1f52680fa3a54bd2983fe38ae400b5d5" member_type: VOTER last_known_addr { host: "127.19.194.3" port: 41377 } } peers { permanent_uuid: "e5ac26f8baab47dd8096af286ff02861" member_type: VOTER last_known_addr { host: "127.19.194.1" port: 42955 } }
I20260806 18:36:53.121364 2996 heartbeater.cc:503] Master 127.19.194.62:40989 was elected leader, sending a full tablet report...
I20260806 18:36:53.121379 3136 heartbeater.cc:503] Master 127.19.194.62:40989 was elected leader, sending a full tablet report...
I20260806 18:36:53.121673 3149 ts_tablet_manager.cc:1428] T 3240790c26d44607b59edcda89b45355 P 1f52680fa3a54bd2983fe38ae400b5d5: Time spent starting tablet: real 0.005s user 0.004s sys 0.001s
I20260806 18:36:53.121675 3150 ts_tablet_manager.cc:1428] T 3240790c26d44607b59edcda89b45355 P 61d7bb0f87d04b1b85aa1dbf30e6bc74: Time spent starting tablet: real 0.005s user 0.005s sys 0.000s
W20260806 18:36:53.125118 2997 tablet.cc:2369] T 3240790c26d44607b59edcda89b45355 P 61d7bb0f87d04b1b85aa1dbf30e6bc74: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260806 18:36:53.178299 2845 tablet.cc:2369] T 3240790c26d44607b59edcda89b45355 P e5ac26f8baab47dd8096af286ff02861: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 18:36:53.215183 2852 raft_consensus.cc:482] T 3240790c26d44607b59edcda89b45355 P e5ac26f8baab47dd8096af286ff02861 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 18:36:53.215490 2852 raft_consensus.cc:504] T 3240790c26d44607b59edcda89b45355 P e5ac26f8baab47dd8096af286ff02861 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "61d7bb0f87d04b1b85aa1dbf30e6bc74" member_type: VOTER last_known_addr { host: "127.19.194.2" port: 40079 } } peers { permanent_uuid: "1f52680fa3a54bd2983fe38ae400b5d5" member_type: VOTER last_known_addr { host: "127.19.194.3" port: 41377 } } peers { permanent_uuid: "e5ac26f8baab47dd8096af286ff02861" member_type: VOTER last_known_addr { host: "127.19.194.1" port: 42955 } }
I20260806 18:36:53.216774 2852 leader_election.cc:288] T 3240790c26d44607b59edcda89b45355 P e5ac26f8baab47dd8096af286ff02861 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 61d7bb0f87d04b1b85aa1dbf30e6bc74 (127.19.194.2:40079), 1f52680fa3a54bd2983fe38ae400b5d5 (127.19.194.3:41377)
I20260806 18:36:53.222033 3147 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:36:53.217159 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.194.1:58909 (local address 127.19.194.3:41377)
0806 18:36:53.217317 (+ 158us) server_negotiation.cc:206] Beginning negotiation
0806 18:36:53.217321 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 18:36:53.217511 (+ 190us) server_negotiation.cc:407] Connection header received
0806 18:36:53.217663 (+ 152us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:36:53.217666 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:36:53.217718 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:36:53.217836 (+ 118us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 18:36:53.218191 (+ 355us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:36:53.219004 (+ 813us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:36:53.220719 (+ 1715us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:36:53.221363 (+ 644us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:36:53.221492 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:36:53.221546 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0806 18:36:53.221855 (+ 309us) server_negotiation.cc:299] Negotiation successful
0806 18:36:53.221921 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260806 18:36:53.222023 3146 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:36:53.217117 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.194.1:40085 (local address 127.19.194.2:40079)
0806 18:36:53.217252 (+ 135us) server_negotiation.cc:206] Beginning negotiation
0806 18:36:53.217257 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 18:36:53.217380 (+ 123us) server_negotiation.cc:407] Connection header received
0806 18:36:53.217543 (+ 163us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:36:53.217546 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:36:53.217599 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:36:53.217686 (+ 87us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 18:36:53.218200 (+ 514us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:36:53.218972 (+ 772us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:36:53.220719 (+ 1747us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:36:53.221370 (+ 651us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:36:53.221499 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:36:53.221546 (+ 47us) server_negotiation.cc:1035] Waiting for connection context
0806 18:36:53.221817 (+ 271us) server_negotiation.cc:299] Negotiation successful
0806 18:36:53.221893 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260806 18:36:53.222108 3156 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:36:53.216969 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.194.2:40079 (local address 127.19.194.1:40085)
0806 18:36:53.217285 (+ 316us) negotiation.cc:107] Waiting for socket to connect
0806 18:36:53.217301 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0806 18:36:53.217429 (+ 128us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:36:53.217871 (+ 442us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:36:53.217874 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:36:53.217893 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 18:36:53.218075 (+ 182us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:36:53.218081 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:36:53.219106 (+ 1025us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:36:53.219109 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:36:53.220583 (+ 1474us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:36:53.220594 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:36:53.221490 (+ 896us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:36:53.221494 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:36:53.221624 (+ 130us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:36:53.221703 (+ 79us) client_negotiation.cc:769] Sending connection context
0806 18:36:53.221785 (+ 82us) client_negotiation.cc:240] Negotiation successful
0806 18:36:53.221890 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":249,"mutex_wait_us":235,"spinlock_wait_cycles":9216,"thread_start_us":98,"threads_started":1}
I20260806 18:36:53.222108 3157 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:36:53.216994 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.194.3:41377 (local address 127.19.194.1:58909)
0806 18:36:53.217426 (+ 432us) negotiation.cc:107] Waiting for socket to connect
0806 18:36:53.217437 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0806 18:36:53.217550 (+ 113us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:36:53.217861 (+ 311us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:36:53.217864 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:36:53.217892 (+ 28us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 18:36:53.218069 (+ 177us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:36:53.218077 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:36:53.219106 (+ 1029us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:36:53.219109 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:36:53.220582 (+ 1473us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:36:53.220594 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:36:53.221490 (+ 896us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:36:53.221494 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:36:53.221622 (+ 128us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:36:53.221703 (+ 81us) client_negotiation.cc:769] Sending connection context
0806 18:36:53.221785 (+ 82us) client_negotiation.cc:240] Negotiation successful
0806 18:36:53.221890 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":379,"mutex_wait_us":200,"spinlock_wait_cycles":1280,"thread_start_us":67,"threads_started":1}
I20260806 18:36:53.222798 2944 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3240790c26d44607b59edcda89b45355" candidate_uuid: "e5ac26f8baab47dd8096af286ff02861" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "61d7bb0f87d04b1b85aa1dbf30e6bc74" is_pre_election: true
I20260806 18:36:53.223114 2944 raft_consensus.cc:2454] T 3240790c26d44607b59edcda89b45355 P 61d7bb0f87d04b1b85aa1dbf30e6bc74 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e5ac26f8baab47dd8096af286ff02861 in term 0.
I20260806 18:36:53.223093 3088 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3240790c26d44607b59edcda89b45355" candidate_uuid: "e5ac26f8baab47dd8096af286ff02861" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1f52680fa3a54bd2983fe38ae400b5d5" is_pre_election: true
I20260806 18:36:53.223444 3088 raft_consensus.cc:2454] T 3240790c26d44607b59edcda89b45355 P 1f52680fa3a54bd2983fe38ae400b5d5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e5ac26f8baab47dd8096af286ff02861 in term 0.
I20260806 18:36:53.223636 2725 leader_election.cc:302] T 3240790c26d44607b59edcda89b45355 P e5ac26f8baab47dd8096af286ff02861 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 61d7bb0f87d04b1b85aa1dbf30e6bc74, e5ac26f8baab47dd8096af286ff02861; no voters:
I20260806 18:36:53.223943 2852 raft_consensus.cc:2781] T 3240790c26d44607b59edcda89b45355 P e5ac26f8baab47dd8096af286ff02861 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 18:36:53.224022 2852 raft_consensus.cc:482] T 3240790c26d44607b59edcda89b45355 P e5ac26f8baab47dd8096af286ff02861 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 18:36:53.224053 2852 raft_consensus.cc:3037] T 3240790c26d44607b59edcda89b45355 P e5ac26f8baab47dd8096af286ff02861 [term 0 FOLLOWER]: Advancing to term 1
I20260806 18:36:53.225137 2852 raft_consensus.cc:504] T 3240790c26d44607b59edcda89b45355 P e5ac26f8baab47dd8096af286ff02861 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "61d7bb0f87d04b1b85aa1dbf30e6bc74" member_type: VOTER last_known_addr { host: "127.19.194.2" port: 40079 } } peers { permanent_uuid: "1f52680fa3a54bd2983fe38ae400b5d5" member_type: VOTER last_known_addr { host: "127.19.194.3" port: 41377 } } peers { permanent_uuid: "e5ac26f8baab47dd8096af286ff02861" member_type: VOTER last_known_addr { host: "127.19.194.1" port: 42955 } }
I20260806 18:36:53.225667 2852 leader_election.cc:288] T 3240790c26d44607b59edcda89b45355 P e5ac26f8baab47dd8096af286ff02861 [CANDIDATE]: Term 1 election: Requested vote from peers 61d7bb0f87d04b1b85aa1dbf30e6bc74 (127.19.194.2:40079), 1f52680fa3a54bd2983fe38ae400b5d5 (127.19.194.3:41377)
I20260806 18:36:53.226017 2944 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3240790c26d44607b59edcda89b45355" candidate_uuid: "e5ac26f8baab47dd8096af286ff02861" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "61d7bb0f87d04b1b85aa1dbf30e6bc74"
I20260806 18:36:53.226028 3088 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3240790c26d44607b59edcda89b45355" candidate_uuid: "e5ac26f8baab47dd8096af286ff02861" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1f52680fa3a54bd2983fe38ae400b5d5"
I20260806 18:36:53.226150 3088 raft_consensus.cc:3037] T 3240790c26d44607b59edcda89b45355 P 1f52680fa3a54bd2983fe38ae400b5d5 [term 0 FOLLOWER]: Advancing to term 1
I20260806 18:36:53.226150 2944 raft_consensus.cc:3037] T 3240790c26d44607b59edcda89b45355 P 61d7bb0f87d04b1b85aa1dbf30e6bc74 [term 0 FOLLOWER]: Advancing to term 1
I20260806 18:36:53.227522 3088 raft_consensus.cc:2454] T 3240790c26d44607b59edcda89b45355 P 1f52680fa3a54bd2983fe38ae400b5d5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e5ac26f8baab47dd8096af286ff02861 in term 1.
I20260806 18:36:53.227522 2944 raft_consensus.cc:2454] T 3240790c26d44607b59edcda89b45355 P 61d7bb0f87d04b1b85aa1dbf30e6bc74 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e5ac26f8baab47dd8096af286ff02861 in term 1.
I20260806 18:36:53.227850 2723 leader_election.cc:302] T 3240790c26d44607b59edcda89b45355 P e5ac26f8baab47dd8096af286ff02861 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1f52680fa3a54bd2983fe38ae400b5d5, e5ac26f8baab47dd8096af286ff02861; no voters:
I20260806 18:36:53.228056 2852 raft_consensus.cc:2781] T 3240790c26d44607b59edcda89b45355 P e5ac26f8baab47dd8096af286ff02861 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 18:36:53.228269 2852 raft_consensus.cc:686] T 3240790c26d44607b59edcda89b45355 P e5ac26f8baab47dd8096af286ff02861 [term 1 LEADER]: Becoming Leader. State: Replica: e5ac26f8baab47dd8096af286ff02861, State: Running, Role: LEADER
I20260806 18:36:53.228533 2852 consensus_queue.cc:237] T 3240790c26d44607b59edcda89b45355 P e5ac26f8baab47dd8096af286ff02861 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "61d7bb0f87d04b1b85aa1dbf30e6bc74" member_type: VOTER last_known_addr { host: "127.19.194.2" port: 40079 } } peers { permanent_uuid: "1f52680fa3a54bd2983fe38ae400b5d5" member_type: VOTER last_known_addr { host: "127.19.194.3" port: 41377 } } peers { permanent_uuid: "e5ac26f8baab47dd8096af286ff02861" member_type: VOTER last_known_addr { host: "127.19.194.1" port: 42955 } }
I20260806 18:36:53.231096 2643 catalog_manager.cc:5495] T 3240790c26d44607b59edcda89b45355 P e5ac26f8baab47dd8096af286ff02861 reported cstate change: term changed from 0 to 1, leader changed from <none> to e5ac26f8baab47dd8096af286ff02861 (127.19.194.1). New cstate: current_term: 1 leader_uuid: "e5ac26f8baab47dd8096af286ff02861" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "61d7bb0f87d04b1b85aa1dbf30e6bc74" member_type: VOTER last_known_addr { host: "127.19.194.2" port: 40079 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1f52680fa3a54bd2983fe38ae400b5d5" member_type: VOTER last_known_addr { host: "127.19.194.3" port: 41377 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e5ac26f8baab47dd8096af286ff02861" member_type: VOTER last_known_addr { host: "127.19.194.1" port: 42955 } health_report { overall_health: HEALTHY } } }
I20260806 18:36:53.252735 3160 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:36:53.249358 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43064 (local address 127.19.194.1:42955)
0806 18:36:53.249626 (+ 268us) server_negotiation.cc:206] Beginning negotiation
0806 18:36:53.249633 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 18:36:53.249645 (+ 12us) server_negotiation.cc:407] Connection header received
0806 18:36:53.249707 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:36:53.249710 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:36:53.249758 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:36:53.249852 (+ 94us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 18:36:53.250074 (+ 222us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:36:53.250618 (+ 544us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:36:53.251483 (+ 865us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:36:53.251716 (+ 233us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:36:53.251791 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:36:53.252054 (+ 263us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 18:36:53.252432 (+ 378us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 18:36:53.252515 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0806 18:36:53.252574 (+ 59us) server_negotiation.cc:299] Negotiation successful
0806 18:36:53.252618 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":208,"thread_start_us":121,"threads_started":1}
I20260806 18:36:53.256448 3088 raft_consensus.cc:1261] T 3240790c26d44607b59edcda89b45355 P 1f52680fa3a54bd2983fe38ae400b5d5 [term 1 FOLLOWER]: Refusing update from remote peer e5ac26f8baab47dd8096af286ff02861: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 18:36:53.256448 2944 raft_consensus.cc:1261] T 3240790c26d44607b59edcda89b45355 P 61d7bb0f87d04b1b85aa1dbf30e6bc74 [term 1 FOLLOWER]: Refusing update from remote peer e5ac26f8baab47dd8096af286ff02861: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 18:36:53.257144 2852 consensus_queue.cc:1035] T 3240790c26d44607b59edcda89b45355 P e5ac26f8baab47dd8096af286ff02861 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1f52680fa3a54bd2983fe38ae400b5d5" member_type: VOTER last_known_addr { host: "127.19.194.3" port: 41377 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 18:36:53.257334 3158 consensus_queue.cc:1035] T 3240790c26d44607b59edcda89b45355 P e5ac26f8baab47dd8096af286ff02861 [LEADER]: Connected to new peer: Peer: permanent_uuid: "61d7bb0f87d04b1b85aa1dbf30e6bc74" member_type: VOTER last_known_addr { host: "127.19.194.2" port: 40079 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 18:36:53.266541 3162 mvcc.cc:205] Tried to move back new op lower bound from 7315625628692312064 to 7315625628586647552. Current Snapshot: MvccSnapshot[applied={T|T < 7315625628692312064}]
I20260806 18:36:53.266844 3163 mvcc.cc:205] Tried to move back new op lower bound from 7315625628692312064 to 7315625628586647552. Current Snapshot: MvccSnapshot[applied={T|T < 7315625628692312064}]
I20260806 18:36:53.273377 2645 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin'} at 127.0.0.1:59920:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 18:36:53.273561 2645 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin'} at 127.0.0.1:59920:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 18:36:53.275890 2645 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 422f52e3e25c47249416f4d5b21c77be: Sending DeleteTablet for 3 replicas of tablet 3240790c26d44607b59edcda89b45355
I20260806 18:36:53.276551 2922 tablet_service.cc:1514] Processing DeleteTablet for tablet 3240790c26d44607b59edcda89b45355 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 18:36:53 UTC) from {username='kudu', principal='kudu/127.19.194.62@KRBTEST.COM'} at 127.0.0.1:50866
I20260806 18:36:53.276791 3066 tablet_service.cc:1514] Processing DeleteTablet for tablet 3240790c26d44607b59edcda89b45355 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 18:36:53 UTC) from {username='kudu', principal='kudu/127.19.194.62@KRBTEST.COM'} at 127.0.0.1:47610
I20260806 18:36:53.276852 2773 tablet_service.cc:1514] Processing DeleteTablet for tablet 3240790c26d44607b59edcda89b45355 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 18:36:53 UTC) from {username='kudu', principal='kudu/127.19.194.62@KRBTEST.COM'} at 127.0.0.1:43048
I20260806 18:36:53.277199 3171 tablet_replica.cc:330] T 3240790c26d44607b59edcda89b45355 P e5ac26f8baab47dd8096af286ff02861: stopping tablet replica
I20260806 18:36:53.277343 3171 raft_consensus.cc:2229] T 3240790c26d44607b59edcda89b45355 P e5ac26f8baab47dd8096af286ff02861 [term 1 LEADER]: Raft consensus shutting down.
I20260806 18:36:53.277346 20232 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJW69tn/build/debug/bin/kudu with pid 2685
I20260806 18:36:53.277634 3171 raft_consensus.cc:2258] T 3240790c26d44607b59edcda89b45355 P e5ac26f8baab47dd8096af286ff02861 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 18:36:53.278245 3172 tablet_replica.cc:330] T 3240790c26d44607b59edcda89b45355 P 1f52680fa3a54bd2983fe38ae400b5d5: stopping tablet replica
I20260806 18:36:53.278528 3172 raft_consensus.cc:2229] T 3240790c26d44607b59edcda89b45355 P 1f52680fa3a54bd2983fe38ae400b5d5 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 18:36:53.278811 3172 raft_consensus.cc:2258] T 3240790c26d44607b59edcda89b45355 P 1f52680fa3a54bd2983fe38ae400b5d5 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 18:36:53.280094 3170 tablet_replica.cc:330] T 3240790c26d44607b59edcda89b45355 P 61d7bb0f87d04b1b85aa1dbf30e6bc74: stopping tablet replica
I20260806 18:36:53.280123 3172 ts_tablet_manager.cc:1905] T 3240790c26d44607b59edcda89b45355 P 1f52680fa3a54bd2983fe38ae400b5d5: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 18:36:53.280367 3170 raft_consensus.cc:2229] T 3240790c26d44607b59edcda89b45355 P 61d7bb0f87d04b1b85aa1dbf30e6bc74 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 18:36:53.280647 3170 raft_consensus.cc:2258] T 3240790c26d44607b59edcda89b45355 P 61d7bb0f87d04b1b85aa1dbf30e6bc74 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 18:36:53.281870 3170 ts_tablet_manager.cc:1905] T 3240790c26d44607b59edcda89b45355 P 61d7bb0f87d04b1b85aa1dbf30e6bc74: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 18:36:53.282945 3172 ts_tablet_manager.cc:1918] T 3240790c26d44607b59edcda89b45355 P 1f52680fa3a54bd2983fe38ae400b5d5: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 18:36:53.283073 3172 log.cc:1196] T 3240790c26d44607b59edcda89b45355 P 1f52680fa3a54bd2983fe38ae400b5d5: Deleting WAL directory at /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786041246094603-20232-0/minicluster-data/ts-2/wal/wals/3240790c26d44607b59edcda89b45355
I20260806 18:36:53.283453 3172 ts_tablet_manager.cc:1939] T 3240790c26d44607b59edcda89b45355 P 1f52680fa3a54bd2983fe38ae400b5d5: Deleting consensus metadata
I20260806 18:36:53.284406 3170 ts_tablet_manager.cc:1918] T 3240790c26d44607b59edcda89b45355 P 61d7bb0f87d04b1b85aa1dbf30e6bc74: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 18:36:53.284510 3170 log.cc:1196] T 3240790c26d44607b59edcda89b45355 P 61d7bb0f87d04b1b85aa1dbf30e6bc74: Deleting WAL directory at /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786041246094603-20232-0/minicluster-data/ts-1/wal/wals/3240790c26d44607b59edcda89b45355
I20260806 18:36:53.284504 2626 catalog_manager.cc:4841] TS 1f52680fa3a54bd2983fe38ae400b5d5 (127.19.194.3:41377): tablet 3240790c26d44607b59edcda89b45355 (table test-table [id=13e6cbf5e7ed42c9bc69da63a11a517d]) successfully deleted
I20260806 18:36:53.284860 3170 ts_tablet_manager.cc:1939] T 3240790c26d44607b59edcda89b45355 P 61d7bb0f87d04b1b85aa1dbf30e6bc74: Deleting consensus metadata
I20260806 18:36:53.285929 2629 catalog_manager.cc:4841] TS 61d7bb0f87d04b1b85aa1dbf30e6bc74 (127.19.194.2:40079): tablet 3240790c26d44607b59edcda89b45355 (table test-table [id=13e6cbf5e7ed42c9bc69da63a11a517d]) successfully deleted
W20260806 18:36:53.287058 2626 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.19.194.1:42955 (error 108)
I20260806 18:36:53.287915 20232 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJW69tn/build/debug/bin/kudu with pid 2850
I20260806 18:36:53.288239 3145 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:36:53.287825 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.194.1:42955 (local address 127.0.0.1:43074)
0806 18:36:53.288020 (+ 195us) negotiation.cc:107] Waiting for socket to connect
0806 18:36:53.288119 (+ 99us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.194.1:42955: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":66}
W20260806 18:36:53.289958 2626 catalog_manager.cc:4584] TS e5ac26f8baab47dd8096af286ff02861 (127.19.194.1:42955): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 3240790c26d44607b59edcda89b45355: Network error: Client connection negotiation failed: client connection to 127.19.194.1:42955: connect: Connection refused (error 111)
I20260806 18:36:53.295305 20232 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJW69tn/build/debug/bin/kudu with pid 2999
I20260806 18:36:53.302340 20232 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJW69tn/build/debug/bin/kudu with pid 2608
I20260806 18:36:53.309899 20232 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJW69tn/build/debug/bin/kudu
/tmp/dist-test-taskJW69tn/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786041246094603-20232-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786041246094603-20232-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786041246094603-20232-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786041246094603-20232-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.19.194.62:40989
--webserver_interface=127.19.194.62
--webserver_port=33889
--keytab_file=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786041246094603-20232-0/krb5kdc/kudu_127.19.194.62.keytab
--principal=kudu/127.19.194.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.19.194.20:38847
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.19.194.62:40989
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786041246094603-20232-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786041246094603-20232-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786041246094603-20232-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 18:36:53.422946 3174 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 18:36:53.423260 3174 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 18:36:53.423305 3174 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 18:36:53.427893 3174 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 18:36:53.427946 3174 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 18:36:53.427964 3174 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 18:36:53.427980 3174 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 18:36:53.428037 3174 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 18:36:53.433079 3174 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.194.20:38847
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786041246094603-20232-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786041246094603-20232-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.19.194.62:40989
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786041246094603-20232-0/krb5kdc/kudu_127.19.194.62.keytab
--principal=kudu/127.19.194.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.19.194.62:40989
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786041246094603-20232-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.19.194.62
--webserver_port=33889
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3174
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786041246094603-20232-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 17:59:13 UTC on ccycloud.quasar-wvsnzx.root.comops.site
build id 4571028
I20260806 18:36:53.434077 3174 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 18:36:53.435134 3174 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 18:36:53.443529 3180 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:36:53.443574 3182 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:36:53.443531 3179 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 18:36:53.443715 3174 server_base.cc:1034] running on GCE node
I20260806 18:36:53.444450 3174 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 18:36:53.445559 3174 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 18:36:53.446748 3174 hybrid_clock.cc:648] HybridClock initialized: now 1786041413446728 us; error 43 us; skew 500 ppm
Aug 06 18:36:53 dist-test-slave-1xrh krb5kdc[2033](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041413, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.194.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 18:36:53.449561 3174 init.cc:377] Logged in from keytab as kudu/127.19.194.62@KRBTEST.COM (short username kudu)
I20260806 18:36:53.450349 3174 webserver.cc:460] Webserver started at http://127.19.194.62:33889/ using document root <none> and password file <none>
I20260806 18:36:53.450884 3174 fs_manager.cc:359] Metadata directory not provided
I20260806 18:36:53.450943 3174 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 18:36:53.454625 3174 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.002s sys 0.000s
I20260806 18:36:53.456705 3189 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 18:36:53.457963 3174 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260806 18:36:53.458072 3174 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786041246094603-20232-0/minicluster-data/master-0/data,/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786041246094603-20232-0/minicluster-data/master-0/wal
uuid: "422f52e3e25c47249416f4d5b21c77be"
format_stamp: "Formatted at 2026-08-06 18:36:51 on dist-test-slave-1xrh"
server_key: "5a4c117b1fc3712dd62d3e9f64694880"
server_key_iv: "78c55a7345d150ac7b0dbed4e7f1b0c6"
server_key_version: "encryptionkey@0"
I20260806 18:36:53.458564 3174 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786041246094603-20232-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786041246094603-20232-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786041246094603-20232-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 18:36:53.471421 3174 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 18:36:53.474622 3174 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 18:36:53.474773 3174 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 18:36:53.481966 3174 rpc_server.cc:307] RPC server started. Bound to: 127.19.194.62:40989
I20260806 18:36:53.482254 3244 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.194.62:40989 every 8 connection(s)
I20260806 18:36:53.483057 3174 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786041246094603-20232-0/minicluster-data/master-0/data/info.pb
I20260806 18:36:53.487210 3245 sys_catalog.cc:263] Verifying existing consensus state
I20260806 18:36:53.487668 20232 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJW69tn/build/debug/bin/kudu as pid 3174
I20260806 18:36:53.488107 20232 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJW69tn/build/debug/bin/kudu
/tmp/dist-test-taskJW69tn/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786041246094603-20232-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786041246094603-20232-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786041246094603-20232-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786041246094603-20232-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.19.194.1:42955
--local_ip_for_outbound_sockets=127.19.194.1
--tserver_master_addrs=127.19.194.62:40989
--webserver_port=38357
--webserver_interface=127.19.194.1
--keytab_file=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786041246094603-20232-0/krb5kdc/kudu_127.19.194.1.keytab
--principal=kudu/127.19.194.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.19.194.20:38847
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786041246094603-20232-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786041246094603-20232-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786041246094603-20232-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260806 18:36:53.488972 3245 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 422f52e3e25c47249416f4d5b21c77be: Bootstrap starting.
I20260806 18:36:53.513758 3245 log.cc:824] T 00000000000000000000000000000000 P 422f52e3e25c47249416f4d5b21c77be: Log is configured to *not* fsync() on all Append() calls
I20260806 18:36:53.528505 3245 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 422f52e3e25c47249416f4d5b21c77be: Bootstrap replayed 1/1 log segments. Stats: ops{read=14 overwritten=0 applied=14 ignored=0} inserts{seen=7 ignored=0} mutations{seen=9 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260806 18:36:53.528939 3245 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 422f52e3e25c47249416f4d5b21c77be: Bootstrap complete.
I20260806 18:36:53.534659 3245 raft_consensus.cc:348] T 00000000000000000000000000000000 P 422f52e3e25c47249416f4d5b21c77be [term 2 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "422f52e3e25c47249416f4d5b21c77be" member_type: VOTER last_known_addr { host: "127.19.194.62" port: 40989 } }
I20260806 18:36:53.535194 3245 raft_consensus.cc:729] T 00000000000000000000000000000000 P 422f52e3e25c47249416f4d5b21c77be [term 2 FOLLOWER]: Becoming Follower/Learner. State: Replica: 422f52e3e25c47249416f4d5b21c77be, State: Initialized, Role: FOLLOWER
I20260806 18:36:53.535760 3245 consensus_queue.cc:260] T 00000000000000000000000000000000 P 422f52e3e25c47249416f4d5b21c77be [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "422f52e3e25c47249416f4d5b21c77be" member_type: VOTER last_known_addr { host: "127.19.194.62" port: 40989 } }
I20260806 18:36:53.535914 3245 raft_consensus.cc:388] T 00000000000000000000000000000000 P 422f52e3e25c47249416f4d5b21c77be [term 2 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 18:36:53.535982 3245 raft_consensus.cc:482] T 00000000000000000000000000000000 P 422f52e3e25c47249416f4d5b21c77be [term 2 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 18:36:53.536093 3245 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 422f52e3e25c47249416f4d5b21c77be [term 2 FOLLOWER]: Advancing to term 3
I20260806 18:36:53.538501 3245 raft_consensus.cc:504] T 00000000000000000000000000000000 P 422f52e3e25c47249416f4d5b21c77be [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "422f52e3e25c47249416f4d5b21c77be" member_type: VOTER last_known_addr { host: "127.19.194.62" port: 40989 } }
I20260806 18:36:53.538847 3245 leader_election.cc:302] T 00000000000000000000000000000000 P 422f52e3e25c47249416f4d5b21c77be [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 422f52e3e25c47249416f4d5b21c77be; no voters:
I20260806 18:36:53.539222 3245 leader_election.cc:288] T 00000000000000000000000000000000 P 422f52e3e25c47249416f4d5b21c77be [CANDIDATE]: Term 3 election: Requested vote from peers
I20260806 18:36:53.539347 3250 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 422f52e3e25c47249416f4d5b21c77be [term 3 FOLLOWER]: Leader election won for term 3
I20260806 18:36:53.539613 3250 raft_consensus.cc:686] T 00000000000000000000000000000000 P 422f52e3e25c47249416f4d5b21c77be [term 3 LEADER]: Becoming Leader. State: Replica: 422f52e3e25c47249416f4d5b21c77be, State: Running, Role: LEADER
I20260806 18:36:53.539911 3250 consensus_queue.cc:237] T 00000000000000000000000000000000 P 422f52e3e25c47249416f4d5b21c77be [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 3, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "422f52e3e25c47249416f4d5b21c77be" member_type: VOTER last_known_addr { host: "127.19.194.62" port: 40989 } }
I20260806 18:36:53.540422 3245 sys_catalog.cc:564] T 00000000000000000000000000000000 P 422f52e3e25c47249416f4d5b21c77be [sys.catalog]: configured and running, proceeding with master startup.
I20260806 18:36:53.541479 3252 sys_catalog.cc:455] T 00000000000000000000000000000000 P 422f52e3e25c47249416f4d5b21c77be [sys.catalog]: SysCatalogTable state changed. Reason: New leader 422f52e3e25c47249416f4d5b21c77be. Latest consensus state: current_term: 3 leader_uuid: "422f52e3e25c47249416f4d5b21c77be" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "422f52e3e25c47249416f4d5b21c77be" member_type: VOTER last_known_addr { host: "127.19.194.62" port: 40989 } } }
I20260806 18:36:53.541483 3251 sys_catalog.cc:455] T 00000000000000000000000000000000 P 422f52e3e25c47249416f4d5b21c77be [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 3 leader_uuid: "422f52e3e25c47249416f4d5b21c77be" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "422f52e3e25c47249416f4d5b21c77be" member_type: VOTER last_known_addr { host: "127.19.194.62" port: 40989 } } }
I20260806 18:36:53.541633 3252 sys_catalog.cc:458] T 00000000000000000000000000000000 P 422f52e3e25c47249416f4d5b21c77be [sys.catalog]: This master's current role is: LEADER
I20260806 18:36:53.541633 3251 sys_catalog.cc:458] T 00000000000000000000000000000000 P 422f52e3e25c47249416f4d5b21c77be [sys.catalog]: This master's current role is: LEADER
I20260806 18:36:53.542168 3260 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 18:36:53.546403 3260 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 18:36:53.546947 3260 catalog_manager.cc:1259] Loaded cluster ID: 4eef91136e8e4ae8b146e3827730dab6
I20260806 18:36:53.547020 3260 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
F20260806 18:36:53.547935 3260 catalog_manager.cc:1466] Initializing Kudu internal certificate authority failed: Runtime error: unable to load data from memory: error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=RSA error:04093004:rsa routines:OLD_RSA_PRIV_DECODE:RSA lib:rsa_ameth.c:121 error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=PKCS8_PRIV_KEY_INFO
*** Check failure stack trace: ***
@ 0x7fa1472579ad google::LogMessage::Fail() at ??:0
@ 0x7fa14725a88a google::LogMessage::SendToLog() at ??:0
@ 0x7fa1472574de google::LogMessage::Flush() at ??:0
@ 0x7fa147258a19 google::LogMessageFatal::~LogMessageFatal() at ??:0
@ 0x7fa151b96fdd _ZZN4kudu6master14CatalogManager24PrepareForLeadershipTaskEvENKUlSt8functionIFNS_6StatusEvEERKNS_9consensus13RaftConsensusElPKcE_clES5_S9_lSB_ at ??:0
@ 0x7fa151b97d2f kudu::master::CatalogManager::PrepareForLeadershipTask() at ??:0
@ 0x7fa151b94579 _ZZN4kudu6master14CatalogManager17ElectedAsLeaderCbEvENKUlvE_clEv at ??:0
@ 0x7fa151bd8640 _ZNSt17_Function_handlerIFvvEZN4kudu6master14CatalogManager17ElectedAsLeaderCbEvEUlvE_E9_M_invokeERKSt9_Any_data at ??:0
@ 0x61d838 std::function<>::operator()() at ??:0
@ 0x7fa1494a93d7 kudu::ThreadPool::DispatchThread() at ??:0
@ 0x7fa1494a9c83 _ZZN4kudu10ThreadPool12CreateThreadEvENKUlvE_clEv at ??:0
@ 0x7fa1494ab3f2 _ZNSt17_Function_handlerIFvvEZN4kudu10ThreadPool12CreateThreadEvEUlvE_E9_M_invokeERKSt9_Any_data at ??:0
@ 0x61d838 std::function<>::operator()() at ??:0
@ 0x7fa14949a512 kudu::Thread::SuperviseThread() at ??:0
@ 0x7fa14aade6db start_thread at ??:0
@ 0x7fa145d5561f clone at ??:0
W20260806 18:36:53.648321 3248 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 18:36:53.648660 3248 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 18:36:53.648725 3248 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 18:36:53.653640 3248 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 18:36:53.653699 3248 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 18:36:53.653756 3248 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.19.194.1
I20260806 18:36:53.659086 3248 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.194.20:38847
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786041246094603-20232-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786041246094603-20232-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786041246094603-20232-0/krb5kdc/kudu_127.19.194.1.keytab
--principal=kudu/127.19.194.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.19.194.1:42955
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786041246094603-20232-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.19.194.1
--webserver_port=38357
--tserver_master_addrs=127.19.194.62:40989
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3248
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.19.194.1
--log_dir=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786041246094603-20232-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 17:59:13 UTC on ccycloud.quasar-wvsnzx.root.comops.site
build id 4571028
I20260806 18:36:53.660152 3248 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 18:36:53.661149 3248 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 18:36:53.670581 3289 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:36:53.670591 3288 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:36:53.670626 3291 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 18:36:53.670809 3248 server_base.cc:1034] running on GCE node
I20260806 18:36:53.671383 3248 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 18:36:53.672034 3248 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 18:36:53.673226 3248 hybrid_clock.cc:648] HybridClock initialized: now 1786041413673210 us; error 27 us; skew 500 ppm
Aug 06 18:36:53 dist-test-slave-1xrh krb5kdc[2033](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041413, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.194.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 18:36:53.676407 3248 init.cc:377] Logged in from keytab as kudu/127.19.194.1@KRBTEST.COM (short username kudu)
I20260806 18:36:53.677249 3248 webserver.cc:460] Webserver started at http://127.19.194.1:38357/ using document root <none> and password file <none>
I20260806 18:36:53.677814 3248 fs_manager.cc:359] Metadata directory not provided
I20260806 18:36:53.677884 3248 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 18:36:53.681898 3248 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.002s sys 0.000s
I20260806 18:36:53.684201 3299 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 18:36:53.685484 3248 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.003s
I20260806 18:36:53.685617 3248 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786041246094603-20232-0/minicluster-data/ts-0/data,/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786041246094603-20232-0/minicluster-data/ts-0/wal
uuid: "e5ac26f8baab47dd8096af286ff02861"
format_stamp: "Formatted at 2026-08-06 18:36:51 on dist-test-slave-1xrh"
server_key: "ecfbc98d4b6d20352ff87523e6e75ace"
server_key_iv: "4d26e6cb9a1633f9b7144c6be6bdbdc2"
server_key_version: "encryptionkey@0"
I20260806 18:36:53.686151 3248 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786041246094603-20232-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786041246094603-20232-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786041246094603-20232-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 18:36:53.699283 3248 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 18:36:53.702348 3248 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 18:36:53.702543 3248 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 18:36:53.703174 3248 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 18:36:53.704286 3308 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260806 18:36:53.709853 3248 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260806 18:36:53.709931 3248 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.006s user 0.000s sys 0.000s
I20260806 18:36:53.710002 3248 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260806 18:36:53.711730 3248 ts_tablet_manager.cc:610] Registered 1 tablets
I20260806 18:36:53.711815 3248 ts_tablet_manager.cc:589] Time spent register tablets: real 0.002s user 0.001s sys 0.000s
I20260806 18:36:53.712083 3308 tablet_bootstrap.cc:492] T 3240790c26d44607b59edcda89b45355 P e5ac26f8baab47dd8096af286ff02861: Bootstrap starting.
I20260806 18:36:53.722374 3248 rpc_server.cc:307] RPC server started. Bound to: 127.19.194.1:42955
I20260806 18:36:53.723430 3248 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786041246094603-20232-0/minicluster-data/ts-0/data/info.pb
I20260806 18:36:53.723599 3420 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.194.1:42955 every 8 connection(s)
I20260806 18:36:53.726783 20232 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJW69tn/build/debug/bin/kudu as pid 3248
I20260806 18:36:53.727351 20232 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJW69tn/build/debug/bin/kudu
/tmp/dist-test-taskJW69tn/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786041246094603-20232-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786041246094603-20232-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786041246094603-20232-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786041246094603-20232-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.19.194.2:40079
--local_ip_for_outbound_sockets=127.19.194.2
--tserver_master_addrs=127.19.194.62:40989
--webserver_port=42471
--webserver_interface=127.19.194.2
--keytab_file=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786041246094603-20232-0/krb5kdc/kudu_127.19.194.2.keytab
--principal=kudu/127.19.194.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.19.194.20:38847
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786041246094603-20232-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786041246094603-20232-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786041246094603-20232-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260806 18:36:53.736156 3308 log.cc:824] T 3240790c26d44607b59edcda89b45355 P e5ac26f8baab47dd8096af286ff02861: Log is configured to *not* fsync() on all Append() calls
I20260806 18:36:53.743891 3308 tablet_bootstrap.cc:492] T 3240790c26d44607b59edcda89b45355 P e5ac26f8baab47dd8096af286ff02861: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260806 18:36:53.744328 3308 tablet_bootstrap.cc:492] T 3240790c26d44607b59edcda89b45355 P e5ac26f8baab47dd8096af286ff02861: Bootstrap complete.
I20260806 18:36:53.745959 3308 ts_tablet_manager.cc:1397] T 3240790c26d44607b59edcda89b45355 P e5ac26f8baab47dd8096af286ff02861: Time spent bootstrapping tablet: real 0.034s user 0.025s sys 0.004s
I20260806 18:36:53.749117 3308 raft_consensus.cc:348] T 3240790c26d44607b59edcda89b45355 P e5ac26f8baab47dd8096af286ff02861 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "61d7bb0f87d04b1b85aa1dbf30e6bc74" member_type: VOTER last_known_addr { host: "127.19.194.2" port: 40079 } } peers { permanent_uuid: "1f52680fa3a54bd2983fe38ae400b5d5" member_type: VOTER last_known_addr { host: "127.19.194.3" port: 41377 } } peers { permanent_uuid: "e5ac26f8baab47dd8096af286ff02861" member_type: VOTER last_known_addr { host: "127.19.194.1" port: 42955 } }
I20260806 18:36:53.749667 3308 raft_consensus.cc:729] T 3240790c26d44607b59edcda89b45355 P e5ac26f8baab47dd8096af286ff02861 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: e5ac26f8baab47dd8096af286ff02861, State: Initialized, Role: FOLLOWER
I20260806 18:36:53.750161 3308 consensus_queue.cc:260] T 3240790c26d44607b59edcda89b45355 P e5ac26f8baab47dd8096af286ff02861 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "61d7bb0f87d04b1b85aa1dbf30e6bc74" member_type: VOTER last_known_addr { host: "127.19.194.2" port: 40079 } } peers { permanent_uuid: "1f52680fa3a54bd2983fe38ae400b5d5" member_type: VOTER last_known_addr { host: "127.19.194.3" port: 41377 } } peers { permanent_uuid: "e5ac26f8baab47dd8096af286ff02861" member_type: VOTER last_known_addr { host: "127.19.194.1" port: 42955 } }
I20260806 18:36:53.751190 3308 ts_tablet_manager.cc:1428] T 3240790c26d44607b59edcda89b45355 P e5ac26f8baab47dd8096af286ff02861: Time spent starting tablet: real 0.005s user 0.008s sys 0.000s
W20260806 18:36:53.795047 3423 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 18:36:53.725839 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.194.62:40989 (local address 127.19.194.1:36681)
0806 18:36:53.726380 (+ 541us) negotiation.cc:107] Waiting for socket to connect
0806 18:36:53.726423 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0806 18:36:53.727532 (+ 1109us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:36:53.794104 (+ 66572us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.194.62:40989: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
Metrics: {"client-negotiator.queue_time_us":331,"thread_start_us":164,"threads_started":1}
W20260806 18:36:53.795625 3421 heartbeater.cc:644] Failed to heartbeat to 127.19.194.62:40989 (0 consecutive failures): Network error: Failed to ping master at 127.19.194.62:40989: Client connection negotiation failed: client connection to 127.19.194.62:40989: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260806 18:36:53.796435 3423 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:36:53.796139 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.194.62:40989 (local address 127.19.194.1:53097)
0806 18:36:53.796288 (+ 149us) negotiation.cc:107] Waiting for socket to connect
0806 18:36:53.796365 (+ 77us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.194.62:40989: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":58}
I20260806 18:36:53.797132 3423 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:36:53.796886 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.194.62:40989 (local address 127.19.194.1:35277)
0806 18:36:53.797019 (+ 133us) negotiation.cc:107] Waiting for socket to connect
0806 18:36:53.797074 (+ 55us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.194.62:40989: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":50}
W20260806 18:36:53.797345 3421 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260806 18:36:53.843412 3424 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 18:36:53.843636 3424 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 18:36:53.843677 3424 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 18:36:53.847705 3424 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 18:36:53.847754 3424 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 18:36:53.847849 3424 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.19.194.2
I20260806 18:36:53.852900 3424 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.194.20:38847
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786041246094603-20232-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786041246094603-20232-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786041246094603-20232-0/krb5kdc/kudu_127.19.194.2.keytab
--principal=kudu/127.19.194.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.19.194.2:40079
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786041246094603-20232-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.19.194.2
--webserver_port=42471
--tserver_master_addrs=127.19.194.62:40989
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3424
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.19.194.2
--log_dir=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786041246094603-20232-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 17:59:13 UTC on ccycloud.quasar-wvsnzx.root.comops.site
build id 4571028
I20260806 18:36:53.853847 3424 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 18:36:53.854784 3424 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260806 18:36:53.864008 3424 server_base.cc:1034] running on GCE node
W20260806 18:36:53.863973 3434 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:36:53.863978 3431 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:36:53.863969 3432 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 18:36:53.864579 3424 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 18:36:53.865150 3424 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 18:36:53.866304 3424 hybrid_clock.cc:648] HybridClock initialized: now 1786041413866289 us; error 29 us; skew 500 ppm
Aug 06 18:36:53 dist-test-slave-1xrh krb5kdc[2033](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041413, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.194.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 18:36:53.868995 3424 init.cc:377] Logged in from keytab as kudu/127.19.194.2@KRBTEST.COM (short username kudu)
I20260806 18:36:53.869834 3424 webserver.cc:460] Webserver started at http://127.19.194.2:42471/ using document root <none> and password file <none>
I20260806 18:36:53.870369 3424 fs_manager.cc:359] Metadata directory not provided
I20260806 18:36:53.870461 3424 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 18:36:53.874078 3424 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.002s sys 0.000s
I20260806 18:36:53.876348 3441 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 18:36:53.877547 3424 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260806 18:36:53.877687 3424 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786041246094603-20232-0/minicluster-data/ts-1/data,/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786041246094603-20232-0/minicluster-data/ts-1/wal
uuid: "61d7bb0f87d04b1b85aa1dbf30e6bc74"
format_stamp: "Formatted at 2026-08-06 18:36:51 on dist-test-slave-1xrh"
server_key: "3aae3b8c41c1da2cab58e82a12f2c908"
server_key_iv: "caba3493fde26304f548f1862222dadb"
server_key_version: "encryptionkey@0"
I20260806 18:36:53.878219 3424 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786041246094603-20232-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786041246094603-20232-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786041246094603-20232-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 18:36:53.896945 3424 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 18:36:53.899780 3424 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 18:36:53.899986 3424 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 18:36:53.900554 3424 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 18:36:53.901329 3424 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 18:36:53.901378 3424 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 18:36:53.901444 3424 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 18:36:53.901481 3424 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 18:36:53.910472 3424 rpc_server.cc:307] RPC server started. Bound to: 127.19.194.2:40079
I20260806 18:36:53.910560 3559 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.194.2:40079 every 8 connection(s)
I20260806 18:36:53.911513 3424 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786041246094603-20232-0/minicluster-data/ts-1/data/info.pb
I20260806 18:36:53.915400 20232 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJW69tn/build/debug/bin/kudu as pid 3424
I20260806 18:36:53.915520 3564 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:36:53.913643 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.194.62:40989 (local address 127.19.194.2:37929)
0806 18:36:53.914122 (+ 479us) negotiation.cc:107] Waiting for socket to connect
0806 18:36:53.914241 (+ 119us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.194.62:40989: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":286,"thread_start_us":130,"threads_started":1}
I20260806 18:36:53.915870 20232 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJW69tn/build/debug/bin/kudu
/tmp/dist-test-taskJW69tn/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786041246094603-20232-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786041246094603-20232-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786041246094603-20232-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786041246094603-20232-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.19.194.3:41377
--local_ip_for_outbound_sockets=127.19.194.3
--tserver_master_addrs=127.19.194.62:40989
--webserver_port=43387
--webserver_interface=127.19.194.3
--keytab_file=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786041246094603-20232-0/krb5kdc/kudu_127.19.194.3.keytab
--principal=kudu/127.19.194.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.19.194.20:38847
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786041246094603-20232-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786041246094603-20232-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786041246094603-20232-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 18:36:53.916112 3561 heartbeater.cc:644] Failed to heartbeat to 127.19.194.62:40989 (0 consecutive failures): Network error: Failed to ping master at 127.19.194.62:40989: Client connection negotiation failed: client connection to 127.19.194.62:40989: connect: Connection refused (error 111)
I20260806 18:36:53.916734 3564 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:36:53.916500 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.194.62:40989 (local address 127.19.194.2:42611)
0806 18:36:53.916617 (+ 117us) negotiation.cc:107] Waiting for socket to connect
0806 18:36:53.916671 (+ 54us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.194.62:40989: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":49}
I20260806 18:36:53.917398 3564 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:36:53.917166 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.194.62:40989 (local address 127.19.194.2:47621)
0806 18:36:53.917266 (+ 100us) negotiation.cc:107] Waiting for socket to connect
0806 18:36:53.917325 (+ 59us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.194.62:40989: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":42}
W20260806 18:36:53.917580 3561 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260806 18:36:54.026818 3565 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 18:36:54.027071 3565 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 18:36:54.027112 3565 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 18:36:54.031019 3565 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 18:36:54.031070 3565 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 18:36:54.031188 3565 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.19.194.3
I20260806 18:36:54.036310 3565 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.194.20:38847
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786041246094603-20232-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786041246094603-20232-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786041246094603-20232-0/krb5kdc/kudu_127.19.194.3.keytab
--principal=kudu/127.19.194.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.19.194.3:41377
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786041246094603-20232-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.19.194.3
--webserver_port=43387
--tserver_master_addrs=127.19.194.62:40989
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3565
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.19.194.3
--log_dir=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786041246094603-20232-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 17:59:13 UTC on ccycloud.quasar-wvsnzx.root.comops.site
build id 4571028
I20260806 18:36:54.037236 3565 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 18:36:54.038137 3565 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 18:36:54.047753 3571 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 18:36:54.047925 3565 server_base.cc:1034] running on GCE node
W20260806 18:36:54.047753 3573 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:36:54.047775 3575 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 18:36:54.048446 3565 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 18:36:54.049067 3565 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 18:36:54.050293 3565 hybrid_clock.cc:648] HybridClock initialized: now 1786041414050273 us; error 29 us; skew 500 ppm
Aug 06 18:36:54 dist-test-slave-1xrh krb5kdc[2033](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041414, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.194.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 18:36:54.053128 3565 init.cc:377] Logged in from keytab as kudu/127.19.194.3@KRBTEST.COM (short username kudu)
I20260806 18:36:54.053957 3565 webserver.cc:460] Webserver started at http://127.19.194.3:43387/ using document root <none> and password file <none>
I20260806 18:36:54.054502 3565 fs_manager.cc:359] Metadata directory not provided
I20260806 18:36:54.054595 3565 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 18:36:54.058305 3565 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.002s sys 0.000s
I20260806 18:36:54.060402 3582 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 18:36:54.061594 3565 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260806 18:36:54.061723 3565 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786041246094603-20232-0/minicluster-data/ts-2/data,/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786041246094603-20232-0/minicluster-data/ts-2/wal
uuid: "1f52680fa3a54bd2983fe38ae400b5d5"
format_stamp: "Formatted at 2026-08-06 18:36:51 on dist-test-slave-1xrh"
server_key: "21c572fb9bd202689166d5ee43308cb2"
server_key_iv: "4916fbff0e77d6686fd222feaf1e5332"
server_key_version: "encryptionkey@0"
I20260806 18:36:54.062245 3565 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786041246094603-20232-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786041246094603-20232-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786041246094603-20232-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 18:36:54.086248 3565 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 18:36:54.089053 3565 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 18:36:54.089287 3565 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 18:36:54.089939 3565 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 18:36:54.090886 3565 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 18:36:54.090965 3565 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 18:36:54.091030 3565 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 18:36:54.091081 3565 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 18:36:54.099704 3565 rpc_server.cc:307] RPC server started. Bound to: 127.19.194.3:41377
I20260806 18:36:54.099733 3699 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.194.3:41377 every 8 connection(s)
I20260806 18:36:54.100628 3565 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786041246094603-20232-0/minicluster-data/ts-2/data/info.pb
I20260806 18:36:54.102720 20232 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJW69tn/build/debug/bin/kudu as pid 3565
I20260806 18:36:54.104820 3702 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:36:54.102789 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.194.62:40989 (local address 127.19.194.3:42739)
0806 18:36:54.103387 (+ 598us) negotiation.cc:107] Waiting for socket to connect
0806 18:36:54.103548 (+ 161us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.194.62:40989: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":351,"thread_start_us":142,"threads_started":1}
W20260806 18:36:54.105274 3700 heartbeater.cc:644] Failed to heartbeat to 127.19.194.62:40989 (0 consecutive failures): Network error: Failed to ping master at 127.19.194.62:40989: Client connection negotiation failed: client connection to 127.19.194.62:40989: connect: Connection refused (error 111)
I20260806 18:36:54.105805 3702 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:36:54.105612 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.194.62:40989 (local address 127.19.194.3:54233)
0806 18:36:54.105708 (+ 96us) negotiation.cc:107] Waiting for socket to connect
0806 18:36:54.105753 (+ 45us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.194.62:40989: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":38}
I20260806 18:36:54.106381 3702 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:36:54.106202 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.194.62:40989 (local address 127.19.194.3:37729)
0806 18:36:54.106289 (+ 87us) negotiation.cc:107] Waiting for socket to connect
0806 18:36:54.106337 (+ 48us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.194.62:40989: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":36}
W20260806 18:36:54.106555 3700 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260806 18:36:54.798560 3704 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:36:54.798079 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.194.62:40989 (local address 127.19.194.1:54887)
0806 18:36:54.798383 (+ 304us) negotiation.cc:107] Waiting for socket to connect
0806 18:36:54.798443 (+ 60us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.194.62:40989: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":229,"thread_start_us":103,"threads_started":1}
I20260806 18:36:54.918488 3706 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:36:54.918097 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.194.62:40989 (local address 127.19.194.2:44247)
0806 18:36:54.918340 (+ 243us) negotiation.cc:107] Waiting for socket to connect
0806 18:36:54.918389 (+ 49us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.194.62:40989: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":180,"thread_start_us":81,"threads_started":1}
I20260806 18:36:55.107539 3707 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:36:55.107037 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.194.62:40989 (local address 127.19.194.3:40045)
0806 18:36:55.107373 (+ 336us) negotiation.cc:107] Waiting for socket to connect
0806 18:36:55.107428 (+ 55us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.194.62:40989: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":247,"thread_start_us":130,"threads_started":1}
I20260806 18:36:55.577277 3708 raft_consensus.cc:482] T 3240790c26d44607b59edcda89b45355 P e5ac26f8baab47dd8096af286ff02861 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 18:36:55.577533 3708 raft_consensus.cc:504] T 3240790c26d44607b59edcda89b45355 P e5ac26f8baab47dd8096af286ff02861 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "61d7bb0f87d04b1b85aa1dbf30e6bc74" member_type: VOTER last_known_addr { host: "127.19.194.2" port: 40079 } } peers { permanent_uuid: "1f52680fa3a54bd2983fe38ae400b5d5" member_type: VOTER last_known_addr { host: "127.19.194.3" port: 41377 } } peers { permanent_uuid: "e5ac26f8baab47dd8096af286ff02861" member_type: VOTER last_known_addr { host: "127.19.194.1" port: 42955 } }
I20260806 18:36:55.578648 3708 leader_election.cc:288] T 3240790c26d44607b59edcda89b45355 P e5ac26f8baab47dd8096af286ff02861 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 61d7bb0f87d04b1b85aa1dbf30e6bc74 (127.19.194.2:40079), 1f52680fa3a54bd2983fe38ae400b5d5 (127.19.194.3:41377)
Aug 06 18:36:55 dist-test-slave-1xrh krb5kdc[2033](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041413, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.194.1@KRBTEST.COM for kudu/127.19.194.3@KRBTEST.COM
Aug 06 18:36:55 dist-test-slave-1xrh krb5kdc[2033](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041413, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.194.1@KRBTEST.COM for kudu/127.19.194.2@KRBTEST.COM
I20260806 18:36:55.595708 3712 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:36:55.578996 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.194.1:35197 (local address 127.19.194.2:40079)
0806 18:36:55.579335 (+ 339us) server_negotiation.cc:206] Beginning negotiation
0806 18:36:55.579352 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0806 18:36:55.579414 (+ 62us) server_negotiation.cc:407] Connection header received
0806 18:36:55.580323 (+ 909us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:36:55.580331 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:36:55.580766 (+ 435us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:36:55.581084 (+ 318us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:36:55.582863 (+ 1779us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:36:55.584460 (+ 1597us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:36:55.586451 (+ 1991us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:36:55.587194 (+ 743us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:36:55.587355 (+ 161us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:36:55.591325 (+ 3970us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:36:55.591331 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:36:55.591344 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 18:36:55.591412 (+ 68us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:36:55.593434 (+ 2022us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:36:55.593914 (+ 480us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:36:55.593920 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:36:55.593927 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:36:55.594059 (+ 132us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:36:55.594354 (+ 295us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:36:55.594356 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:36:55.594358 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:36:55.594794 (+ 436us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:36:55.594926 (+ 132us) server_negotiation.cc:1035] Waiting for connection context
0806 18:36:55.595264 (+ 338us) server_negotiation.cc:299] Negotiation successful
0806 18:36:55.595542 (+ 278us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":207,"thread_start_us":60,"threads_started":1}
I20260806 18:36:55.595686 3711 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:36:55.578907 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.194.3:41377 (local address 127.19.194.1:41765)
0806 18:36:55.579391 (+ 484us) negotiation.cc:107] Waiting for socket to connect
0806 18:36:55.579403 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0806 18:36:55.579476 (+ 73us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:36:55.581187 (+ 1711us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:36:55.581194 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:36:55.581812 (+ 618us) client_negotiation.cc:189] Negotiated authn=SASL
0806 18:36:55.582677 (+ 865us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:36:55.582699 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:36:55.584627 (+ 1928us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:36:55.584631 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:36:55.586320 (+ 1689us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:36:55.586331 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:36:55.587540 (+ 1209us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:36:55.587544 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:36:55.587826 (+ 282us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:36:55.588551 (+ 725us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 18:36:55.588580 (+ 29us) client_negotiation.cc:656] calling sasl_client_start()
0806 18:36:55.590613 (+ 2033us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 18:36:55.592924 (+ 2311us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:36:55.592931 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:36:55.592944 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:36:55.593283 (+ 339us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:36:55.593711 (+ 428us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:36:55.593713 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:36:55.593714 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:36:55.593866 (+ 152us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:36:55.594636 (+ 770us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 18:36:55.594642 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 18:36:55.594892 (+ 250us) client_negotiation.cc:769] Sending connection context
0806 18:36:55.595125 (+ 233us) client_negotiation.cc:240] Negotiation successful
0806 18:36:55.595492 (+ 367us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":430,"mutex_wait_us":396,"spinlock_wait_cycles":14720,"thread_start_us":65,"threads_started":1}
I20260806 18:36:55.595801 3710 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:36:55.578795 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.194.2:40079 (local address 127.19.194.1:35197)
0806 18:36:55.579261 (+ 466us) negotiation.cc:107] Waiting for socket to connect
0806 18:36:55.579277 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0806 18:36:55.579391 (+ 114us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:36:55.581188 (+ 1797us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:36:55.581194 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:36:55.581811 (+ 617us) client_negotiation.cc:189] Negotiated authn=SASL
0806 18:36:55.582677 (+ 866us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:36:55.582699 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:36:55.584627 (+ 1928us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:36:55.584631 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:36:55.586320 (+ 1689us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:36:55.586331 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:36:55.587540 (+ 1209us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:36:55.587544 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:36:55.587823 (+ 279us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:36:55.588551 (+ 728us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 18:36:55.588580 (+ 29us) client_negotiation.cc:656] calling sasl_client_start()
0806 18:36:55.591088 (+ 2508us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 18:36:55.593552 (+ 2464us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:36:55.593556 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:36:55.593558 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:36:55.593808 (+ 250us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:36:55.594158 (+ 350us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:36:55.594161 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:36:55.594163 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:36:55.594262 (+ 99us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:36:55.594904 (+ 642us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 18:36:55.594906 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 18:36:55.594987 (+ 81us) client_negotiation.cc:769] Sending connection context
0806 18:36:55.595124 (+ 137us) client_negotiation.cc:240] Negotiation successful
0806 18:36:55.595589 (+ 465us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":399,"mutex_wait_us":570,"spinlock_wait_cycles":59392,"thread_start_us":93,"threads_started":1}
I20260806 18:36:55.595755 3713 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:36:55.579088 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.194.1:41765 (local address 127.19.194.3:41377)
0806 18:36:55.579507 (+ 419us) server_negotiation.cc:206] Beginning negotiation
0806 18:36:55.579526 (+ 19us) server_negotiation.cc:399] Waiting for connection header
0806 18:36:55.579556 (+ 30us) server_negotiation.cc:407] Connection header received
0806 18:36:55.580322 (+ 766us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:36:55.580331 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:36:55.580766 (+ 435us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:36:55.581084 (+ 318us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:36:55.582864 (+ 1780us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:36:55.584460 (+ 1596us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:36:55.586451 (+ 1991us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:36:55.587194 (+ 743us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:36:55.587355 (+ 161us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:36:55.590920 (+ 3565us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:36:55.590927 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:36:55.590943 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 18:36:55.590979 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:36:55.592789 (+ 1810us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:36:55.593424 (+ 635us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:36:55.593429 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:36:55.593433 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:36:55.593596 (+ 163us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:36:55.593979 (+ 383us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:36:55.593982 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:36:55.593983 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:36:55.594506 (+ 523us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:36:55.594604 (+ 98us) server_negotiation.cc:1035] Waiting for connection context
0806 18:36:55.595276 (+ 672us) server_negotiation.cc:299] Negotiation successful
0806 18:36:55.595558 (+ 282us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":326,"thread_start_us":103,"threads_started":1}
W20260806 18:36:55.598409 3300 leader_election.cc:341] T 3240790c26d44607b59edcda89b45355 P e5ac26f8baab47dd8096af286ff02861 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 1f52680fa3a54bd2983fe38ae400b5d5 (127.19.194.3:41377): Not found: Tablet not found: 3240790c26d44607b59edcda89b45355
W20260806 18:36:55.598567 3303 leader_election.cc:341] T 3240790c26d44607b59edcda89b45355 P e5ac26f8baab47dd8096af286ff02861 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 61d7bb0f87d04b1b85aa1dbf30e6bc74 (127.19.194.2:40079): Not found: Tablet not found: 3240790c26d44607b59edcda89b45355
I20260806 18:36:55.598677 3303 leader_election.cc:302] T 3240790c26d44607b59edcda89b45355 P e5ac26f8baab47dd8096af286ff02861 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e5ac26f8baab47dd8096af286ff02861; no voters: 1f52680fa3a54bd2983fe38ae400b5d5, 61d7bb0f87d04b1b85aa1dbf30e6bc74
I20260806 18:36:55.598961 3708 raft_consensus.cc:2726] T 3240790c26d44607b59edcda89b45355 P e5ac26f8baab47dd8096af286ff02861 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260806 18:36:55.799644 3710 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:36:55.799326 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.194.62:40989 (local address 127.19.194.1:59683)
0806 18:36:55.799481 (+ 155us) negotiation.cc:107] Waiting for socket to connect
0806 18:36:55.799546 (+ 65us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.194.62:40989: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":56}
I20260806 18:36:55.919628 3714 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:36:55.919209 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.194.62:40989 (local address 127.19.194.2:56325)
0806 18:36:55.919458 (+ 249us) negotiation.cc:107] Waiting for socket to connect
0806 18:36:55.919510 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.194.62:40989: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":181,"thread_start_us":90,"threads_started":1}
I20260806 18:36:56.108613 3715 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:36:56.108230 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.194.62:40989 (local address 127.19.194.3:41365)
0806 18:36:56.108462 (+ 232us) negotiation.cc:107] Waiting for socket to connect
0806 18:36:56.108516 (+ 54us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.194.62:40989: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":161,"thread_start_us":63,"threads_started":1}
I20260806 18:36:56.800920 3716 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:36:56.800474 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.194.62:40989 (local address 127.19.194.1:34463)
0806 18:36:56.800763 (+ 289us) negotiation.cc:107] Waiting for socket to connect
0806 18:36:56.800819 (+ 56us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.194.62:40989: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":217,"thread_start_us":105,"threads_started":1}
I20260806 18:36:56.920665 3717 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:36:56.920294 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.194.62:40989 (local address 127.19.194.2:42775)
0806 18:36:56.920529 (+ 235us) negotiation.cc:107] Waiting for socket to connect
0806 18:36:56.920579 (+ 50us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.194.62:40989: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":170,"thread_start_us":78,"threads_started":1}
I20260806 18:36:57.109797 3718 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:36:57.109429 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.194.62:40989 (local address 127.19.194.3:33141)
0806 18:36:57.109670 (+ 241us) negotiation.cc:107] Waiting for socket to connect
0806 18:36:57.109720 (+ 50us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.194.62:40989: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":166,"thread_start_us":72,"threads_started":1}
I20260806 18:36:57.524469 3719 raft_consensus.cc:482] T 3240790c26d44607b59edcda89b45355 P e5ac26f8baab47dd8096af286ff02861 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 18:36:57.524617 3719 raft_consensus.cc:504] T 3240790c26d44607b59edcda89b45355 P e5ac26f8baab47dd8096af286ff02861 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "61d7bb0f87d04b1b85aa1dbf30e6bc74" member_type: VOTER last_known_addr { host: "127.19.194.2" port: 40079 } } peers { permanent_uuid: "1f52680fa3a54bd2983fe38ae400b5d5" member_type: VOTER last_known_addr { host: "127.19.194.3" port: 41377 } } peers { permanent_uuid: "e5ac26f8baab47dd8096af286ff02861" member_type: VOTER last_known_addr { host: "127.19.194.1" port: 42955 } }
I20260806 18:36:57.525089 3719 leader_election.cc:288] T 3240790c26d44607b59edcda89b45355 P e5ac26f8baab47dd8096af286ff02861 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 61d7bb0f87d04b1b85aa1dbf30e6bc74 (127.19.194.2:40079), 1f52680fa3a54bd2983fe38ae400b5d5 (127.19.194.3:41377)
W20260806 18:36:57.525902 3300 leader_election.cc:341] T 3240790c26d44607b59edcda89b45355 P e5ac26f8baab47dd8096af286ff02861 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 1f52680fa3a54bd2983fe38ae400b5d5 (127.19.194.3:41377): Not found: Tablet not found: 3240790c26d44607b59edcda89b45355
W20260806 18:36:57.525992 3303 leader_election.cc:341] T 3240790c26d44607b59edcda89b45355 P e5ac26f8baab47dd8096af286ff02861 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 61d7bb0f87d04b1b85aa1dbf30e6bc74 (127.19.194.2:40079): Not found: Tablet not found: 3240790c26d44607b59edcda89b45355
I20260806 18:36:57.526113 3303 leader_election.cc:302] T 3240790c26d44607b59edcda89b45355 P e5ac26f8baab47dd8096af286ff02861 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e5ac26f8baab47dd8096af286ff02861; no voters: 1f52680fa3a54bd2983fe38ae400b5d5, 61d7bb0f87d04b1b85aa1dbf30e6bc74
I20260806 18:36:57.526311 3719 raft_consensus.cc:2726] T 3240790c26d44607b59edcda89b45355 P e5ac26f8baab47dd8096af286ff02861 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260806 18:36:57.802183 3720 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:36:57.801707 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.194.62:40989 (local address 127.19.194.1:41327)
0806 18:36:57.802010 (+ 303us) negotiation.cc:107] Waiting for socket to connect
0806 18:36:57.802068 (+ 58us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.194.62:40989: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":232,"thread_start_us":94,"threads_started":1}
I20260806 18:36:57.921742 3721 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:36:57.921324 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.194.62:40989 (local address 127.19.194.2:60293)
0806 18:36:57.921571 (+ 247us) negotiation.cc:107] Waiting for socket to connect
0806 18:36:57.921637 (+ 66us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.194.62:40989: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":170,"thread_start_us":72,"threads_started":1}
I20260806 18:36:58.110960 3722 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:36:58.110530 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.194.62:40989 (local address 127.19.194.3:52149)
0806 18:36:58.110786 (+ 256us) negotiation.cc:107] Waiting for socket to connect
0806 18:36:58.110874 (+ 88us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.194.62:40989: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":192,"thread_start_us":86,"threads_started":1}
I20260806 18:36:58.803849 3723 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:36:58.803337 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.194.62:40989 (local address 127.19.194.1:37959)
0806 18:36:58.803639 (+ 302us) negotiation.cc:107] Waiting for socket to connect
0806 18:36:58.803693 (+ 54us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.194.62:40989: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":232,"thread_start_us":79,"threads_started":1}
I20260806 18:36:58.922915 3724 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:36:58.922475 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.194.62:40989 (local address 127.19.194.2:40153)
0806 18:36:58.922741 (+ 266us) negotiation.cc:107] Waiting for socket to connect
0806 18:36:58.922813 (+ 72us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.194.62:40989: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":166,"thread_start_us":72,"threads_started":1}
I20260806 18:36:59.112035 3725 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:36:59.111629 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.194.62:40989 (local address 127.19.194.3:46717)
0806 18:36:59.111891 (+ 262us) negotiation.cc:107] Waiting for socket to connect
0806 18:36:59.111945 (+ 54us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.194.62:40989: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":189,"thread_start_us":87,"threads_started":1}
I20260806 18:36:59.150985 3726 raft_consensus.cc:482] T 3240790c26d44607b59edcda89b45355 P e5ac26f8baab47dd8096af286ff02861 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 18:36:59.151108 3726 raft_consensus.cc:504] T 3240790c26d44607b59edcda89b45355 P e5ac26f8baab47dd8096af286ff02861 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "61d7bb0f87d04b1b85aa1dbf30e6bc74" member_type: VOTER last_known_addr { host: "127.19.194.2" port: 40079 } } peers { permanent_uuid: "1f52680fa3a54bd2983fe38ae400b5d5" member_type: VOTER last_known_addr { host: "127.19.194.3" port: 41377 } } peers { permanent_uuid: "e5ac26f8baab47dd8096af286ff02861" member_type: VOTER last_known_addr { host: "127.19.194.1" port: 42955 } }
I20260806 18:36:59.151625 3726 leader_election.cc:288] T 3240790c26d44607b59edcda89b45355 P e5ac26f8baab47dd8096af286ff02861 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 61d7bb0f87d04b1b85aa1dbf30e6bc74 (127.19.194.2:40079), 1f52680fa3a54bd2983fe38ae400b5d5 (127.19.194.3:41377)
W20260806 18:36:59.152465 3303 leader_election.cc:341] T 3240790c26d44607b59edcda89b45355 P e5ac26f8baab47dd8096af286ff02861 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 61d7bb0f87d04b1b85aa1dbf30e6bc74 (127.19.194.2:40079): Not found: Tablet not found: 3240790c26d44607b59edcda89b45355
W20260806 18:36:59.152585 3300 leader_election.cc:341] T 3240790c26d44607b59edcda89b45355 P e5ac26f8baab47dd8096af286ff02861 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 1f52680fa3a54bd2983fe38ae400b5d5 (127.19.194.3:41377): Not found: Tablet not found: 3240790c26d44607b59edcda89b45355
I20260806 18:36:59.152652 3300 leader_election.cc:302] T 3240790c26d44607b59edcda89b45355 P e5ac26f8baab47dd8096af286ff02861 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e5ac26f8baab47dd8096af286ff02861; no voters: 1f52680fa3a54bd2983fe38ae400b5d5, 61d7bb0f87d04b1b85aa1dbf30e6bc74
I20260806 18:36:59.152786 3726 raft_consensus.cc:2726] T 3240790c26d44607b59edcda89b45355 P e5ac26f8baab47dd8096af286ff02861 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260806 18:36:59.805106 3727 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:36:59.804660 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.194.62:40989 (local address 127.19.194.1:38249)
0806 18:36:59.804931 (+ 271us) negotiation.cc:107] Waiting for socket to connect
0806 18:36:59.805005 (+ 74us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.194.62:40989: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":194,"thread_start_us":78,"threads_started":1}
I20260806 18:36:59.924085 3728 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:36:59.923672 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.194.62:40989 (local address 127.19.194.2:36973)
0806 18:36:59.923929 (+ 257us) negotiation.cc:107] Waiting for socket to connect
0806 18:36:59.923986 (+ 57us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.194.62:40989: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":186,"thread_start_us":74,"threads_started":1}
I20260806 18:37:00.113224 3729 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:37:00.112800 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.194.62:40989 (local address 127.19.194.3:43165)
0806 18:37:00.113047 (+ 247us) negotiation.cc:107] Waiting for socket to connect
0806 18:37:00.113101 (+ 54us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.194.62:40989: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":172,"thread_start_us":75,"threads_started":1}
I20260806 18:37:00.806354 3730 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:37:00.805852 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.194.62:40989 (local address 127.19.194.1:41131)
0806 18:37:00.806140 (+ 288us) negotiation.cc:107] Waiting for socket to connect
0806 18:37:00.806197 (+ 57us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.194.62:40989: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":214,"thread_start_us":78,"threads_started":1}
I20260806 18:37:00.925277 3731 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:37:00.924835 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.194.62:40989 (local address 127.19.194.2:44429)
0806 18:37:00.925127 (+ 292us) negotiation.cc:107] Waiting for socket to connect
0806 18:37:00.925180 (+ 53us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.194.62:40989: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":225,"thread_start_us":73,"threads_started":1}
I20260806 18:37:01.114514 3732 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:37:01.114068 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.194.62:40989 (local address 127.19.194.3:42143)
0806 18:37:01.114374 (+ 306us) negotiation.cc:107] Waiting for socket to connect
0806 18:37:01.114429 (+ 55us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.194.62:40989: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":240,"thread_start_us":87,"threads_started":1}
I20260806 18:37:01.168641 3733 raft_consensus.cc:482] T 3240790c26d44607b59edcda89b45355 P e5ac26f8baab47dd8096af286ff02861 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 18:37:01.168771 3733 raft_consensus.cc:504] T 3240790c26d44607b59edcda89b45355 P e5ac26f8baab47dd8096af286ff02861 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "61d7bb0f87d04b1b85aa1dbf30e6bc74" member_type: VOTER last_known_addr { host: "127.19.194.2" port: 40079 } } peers { permanent_uuid: "1f52680fa3a54bd2983fe38ae400b5d5" member_type: VOTER last_known_addr { host: "127.19.194.3" port: 41377 } } peers { permanent_uuid: "e5ac26f8baab47dd8096af286ff02861" member_type: VOTER last_known_addr { host: "127.19.194.1" port: 42955 } }
I20260806 18:37:01.169296 3733 leader_election.cc:288] T 3240790c26d44607b59edcda89b45355 P e5ac26f8baab47dd8096af286ff02861 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 61d7bb0f87d04b1b85aa1dbf30e6bc74 (127.19.194.2:40079), 1f52680fa3a54bd2983fe38ae400b5d5 (127.19.194.3:41377)
W20260806 18:37:01.170078 3303 leader_election.cc:341] T 3240790c26d44607b59edcda89b45355 P e5ac26f8baab47dd8096af286ff02861 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 61d7bb0f87d04b1b85aa1dbf30e6bc74 (127.19.194.2:40079): Not found: Tablet not found: 3240790c26d44607b59edcda89b45355
W20260806 18:37:01.170183 3300 leader_election.cc:341] T 3240790c26d44607b59edcda89b45355 P e5ac26f8baab47dd8096af286ff02861 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 1f52680fa3a54bd2983fe38ae400b5d5 (127.19.194.3:41377): Not found: Tablet not found: 3240790c26d44607b59edcda89b45355
I20260806 18:37:01.170248 3300 leader_election.cc:302] T 3240790c26d44607b59edcda89b45355 P e5ac26f8baab47dd8096af286ff02861 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e5ac26f8baab47dd8096af286ff02861; no voters: 1f52680fa3a54bd2983fe38ae400b5d5, 61d7bb0f87d04b1b85aa1dbf30e6bc74
I20260806 18:37:01.170396 3733 raft_consensus.cc:2726] T 3240790c26d44607b59edcda89b45355 P e5ac26f8baab47dd8096af286ff02861 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260806 18:37:01.807482 3734 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:37:01.807073 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.194.62:40989 (local address 127.19.194.1:45117)
0806 18:37:01.807331 (+ 258us) negotiation.cc:107] Waiting for socket to connect
0806 18:37:01.807382 (+ 51us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.194.62:40989: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":190,"thread_start_us":98,"threads_started":1}
I20260806 18:37:01.926385 3735 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:37:01.925962 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.194.62:40989 (local address 127.19.194.2:47447)
0806 18:37:01.926234 (+ 272us) negotiation.cc:107] Waiting for socket to connect
0806 18:37:01.926288 (+ 54us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.194.62:40989: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":201,"thread_start_us":88,"threads_started":1}
I20260806 18:37:02.115643 3736 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:37:02.115265 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.194.62:40989 (local address 127.19.194.3:52033)
0806 18:37:02.115495 (+ 230us) negotiation.cc:107] Waiting for socket to connect
0806 18:37:02.115557 (+ 62us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.194.62:40989: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":166,"thread_start_us":72,"threads_started":1}
I20260806 18:37:02.808851 3737 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:37:02.808248 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.194.62:40989 (local address 127.19.194.1:37747)
0806 18:37:02.808676 (+ 428us) negotiation.cc:107] Waiting for socket to connect
0806 18:37:02.808750 (+ 74us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.194.62:40989: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":331,"thread_start_us":113,"threads_started":1}
I20260806 18:37:02.927589 3738 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:37:02.927182 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.194.62:40989 (local address 127.19.194.2:37023)
0806 18:37:02.927440 (+ 258us) negotiation.cc:107] Waiting for socket to connect
0806 18:37:02.927494 (+ 54us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.194.62:40989: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":188,"thread_start_us":71,"threads_started":1}
I20260806 18:37:03.116814 3739 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:37:03.116413 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.194.62:40989 (local address 127.19.194.3:35895)
0806 18:37:03.116665 (+ 252us) negotiation.cc:107] Waiting for socket to connect
0806 18:37:03.116721 (+ 56us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.194.62:40989: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":178,"thread_start_us":73,"threads_started":1}
I20260806 18:37:03.810148 3740 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:37:03.809685 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.194.62:40989 (local address 127.19.194.1:37027)
0806 18:37:03.809983 (+ 298us) negotiation.cc:107] Waiting for socket to connect
0806 18:37:03.810049 (+ 66us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.194.62:40989: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":228,"thread_start_us":78,"threads_started":1}
I20260806 18:37:03.928689 3741 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:37:03.928296 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.194.62:40989 (local address 127.19.194.2:53191)
0806 18:37:03.928544 (+ 248us) negotiation.cc:107] Waiting for socket to connect
0806 18:37:03.928594 (+ 50us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.194.62:40989: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":179,"thread_start_us":72,"threads_started":1}
I20260806 18:37:04.117924 3742 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:37:04.117512 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.194.62:40989 (local address 127.19.194.3:57121)
0806 18:37:04.117776 (+ 264us) negotiation.cc:107] Waiting for socket to connect
0806 18:37:04.117834 (+ 58us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.194.62:40989: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":197,"thread_start_us":69,"threads_started":1}
I20260806 18:37:04.811367 3743 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:37:04.810884 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.194.62:40989 (local address 127.19.194.1:53539)
0806 18:37:04.811210 (+ 326us) negotiation.cc:107] Waiting for socket to connect
0806 18:37:04.811265 (+ 55us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.194.62:40989: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":192,"thread_start_us":90,"threads_started":1}
I20260806 18:37:04.929841 3744 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:37:04.929437 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.194.62:40989 (local address 127.19.194.2:34113)
0806 18:37:04.929693 (+ 256us) negotiation.cc:107] Waiting for socket to connect
0806 18:37:04.929745 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.194.62:40989: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":187,"thread_start_us":80,"threads_started":1}
I20260806 18:37:05.119210 3745 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:37:05.118713 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.194.62:40989 (local address 127.19.194.3:46749)
0806 18:37:05.118984 (+ 271us) negotiation.cc:107] Waiting for socket to connect
0806 18:37:05.119063 (+ 79us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.194.62:40989: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":202,"thread_start_us":91,"threads_started":1}
I20260806 18:37:05.812474 3746 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:37:05.812083 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.194.62:40989 (local address 127.19.194.1:58973)
0806 18:37:05.812322 (+ 239us) negotiation.cc:107] Waiting for socket to connect
0806 18:37:05.812376 (+ 54us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.194.62:40989: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":172,"thread_start_us":76,"threads_started":1}
I20260806 18:37:05.930986 3747 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:37:05.930554 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.194.62:40989 (local address 127.19.194.2:59873)
0806 18:37:05.930815 (+ 261us) negotiation.cc:107] Waiting for socket to connect
0806 18:37:05.930889 (+ 74us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.194.62:40989: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":189,"thread_start_us":86,"threads_started":1}
I20260806 18:37:06.120281 3748 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:37:06.119873 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.194.62:40989 (local address 127.19.194.3:56939)
0806 18:37:06.120128 (+ 255us) negotiation.cc:107] Waiting for socket to connect
0806 18:37:06.120191 (+ 63us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.194.62:40989: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":188,"thread_start_us":87,"threads_started":1}
I20260806 18:37:06.813602 3749 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:37:06.813197 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.194.62:40989 (local address 127.19.194.1:59883)
0806 18:37:06.813452 (+ 255us) negotiation.cc:107] Waiting for socket to connect
0806 18:37:06.813505 (+ 53us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.194.62:40989: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":184,"thread_start_us":74,"threads_started":1}
I20260806 18:37:06.932204 3750 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:37:06.931749 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.194.62:40989 (local address 127.19.194.2:59375)
0806 18:37:06.932048 (+ 299us) negotiation.cc:107] Waiting for socket to connect
0806 18:37:06.932103 (+ 55us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.194.62:40989: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":219,"thread_start_us":114,"threads_started":1}
I20260806 18:37:07.121405 3751 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:37:07.120989 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.194.62:40989 (local address 127.19.194.3:36689)
0806 18:37:07.121255 (+ 266us) negotiation.cc:107] Waiting for socket to connect
0806 18:37:07.121311 (+ 56us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.194.62:40989: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":199,"thread_start_us":103,"threads_started":1}
I20260806 18:37:07.814916 3752 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:37:07.814403 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.194.62:40989 (local address 127.19.194.1:33775)
0806 18:37:07.814754 (+ 351us) negotiation.cc:107] Waiting for socket to connect
0806 18:37:07.814813 (+ 59us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.194.62:40989: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":277,"thread_start_us":72,"threads_started":1}
I20260806 18:37:07.933384 3753 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:37:07.932965 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.194.62:40989 (local address 127.19.194.2:54319)
0806 18:37:07.933232 (+ 267us) negotiation.cc:107] Waiting for socket to connect
0806 18:37:07.933286 (+ 54us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.194.62:40989: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":199,"thread_start_us":101,"threads_started":1}
I20260806 18:37:08.122512 3754 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:37:08.122096 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.194.62:40989 (local address 127.19.194.3:58511)
0806 18:37:08.122356 (+ 260us) negotiation.cc:107] Waiting for socket to connect
0806 18:37:08.122413 (+ 57us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.194.62:40989: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":184,"thread_start_us":75,"threads_started":1}
I20260806 18:37:08.721583 3755 raft_consensus.cc:482] T 3240790c26d44607b59edcda89b45355 P e5ac26f8baab47dd8096af286ff02861 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 18:37:08.721719 3755 raft_consensus.cc:504] T 3240790c26d44607b59edcda89b45355 P e5ac26f8baab47dd8096af286ff02861 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "61d7bb0f87d04b1b85aa1dbf30e6bc74" member_type: VOTER last_known_addr { host: "127.19.194.2" port: 40079 } } peers { permanent_uuid: "1f52680fa3a54bd2983fe38ae400b5d5" member_type: VOTER last_known_addr { host: "127.19.194.3" port: 41377 } } peers { permanent_uuid: "e5ac26f8baab47dd8096af286ff02861" member_type: VOTER last_known_addr { host: "127.19.194.1" port: 42955 } }
I20260806 18:37:08.722180 3755 leader_election.cc:288] T 3240790c26d44607b59edcda89b45355 P e5ac26f8baab47dd8096af286ff02861 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 61d7bb0f87d04b1b85aa1dbf30e6bc74 (127.19.194.2:40079), 1f52680fa3a54bd2983fe38ae400b5d5 (127.19.194.3:41377)
W20260806 18:37:08.723024 3300 leader_election.cc:341] T 3240790c26d44607b59edcda89b45355 P e5ac26f8baab47dd8096af286ff02861 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 1f52680fa3a54bd2983fe38ae400b5d5 (127.19.194.3:41377): Not found: Tablet not found: 3240790c26d44607b59edcda89b45355
W20260806 18:37:08.723121 3303 leader_election.cc:341] T 3240790c26d44607b59edcda89b45355 P e5ac26f8baab47dd8096af286ff02861 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 61d7bb0f87d04b1b85aa1dbf30e6bc74 (127.19.194.2:40079): Not found: Tablet not found: 3240790c26d44607b59edcda89b45355
I20260806 18:37:08.723255 3303 leader_election.cc:302] T 3240790c26d44607b59edcda89b45355 P e5ac26f8baab47dd8096af286ff02861 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e5ac26f8baab47dd8096af286ff02861; no voters: 1f52680fa3a54bd2983fe38ae400b5d5, 61d7bb0f87d04b1b85aa1dbf30e6bc74
I20260806 18:37:08.723397 3755 raft_consensus.cc:2726] T 3240790c26d44607b59edcda89b45355 P e5ac26f8baab47dd8096af286ff02861 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260806 18:37:08.816277 3756 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:37:08.815886 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.194.62:40989 (local address 127.19.194.1:39005)
0806 18:37:08.816127 (+ 241us) negotiation.cc:107] Waiting for socket to connect
0806 18:37:08.816181 (+ 54us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.194.62:40989: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":174,"thread_start_us":74,"threads_started":1}
I20260806 18:37:08.934604 3757 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:37:08.934197 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.194.62:40989 (local address 127.19.194.2:42687)
0806 18:37:08.934457 (+ 260us) negotiation.cc:107] Waiting for socket to connect
0806 18:37:08.934509 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.194.62:40989: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":191,"thread_start_us":84,"threads_started":1}
I20260806 18:37:09.107003 20232 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJW69tn/build/debug/bin/kudu with pid 3248
I20260806 18:37:09.114637 20232 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJW69tn/build/debug/bin/kudu with pid 3424
I20260806 18:37:09.121246 20232 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJW69tn/build/debug/bin/kudu with pid 3565
2026-08-06T18:37:09Z chronyd exiting
[ OK ] SecurityITest.TestEnableAndDisableEncryptedIPKICert (19326 ms)
[ RUN ] SecurityITest.TestEnableAndDisableEncryptedTSK
Loading random data
Initializing database '/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786041246094603-20232-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 18:37:09 dist-test-slave-1xrh krb5kdc[3760](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 18:37:09 dist-test-slave-1xrh krb5kdc[3760](info): set up 2 sockets
Aug 06 18:37:09 dist-test-slave-1xrh krb5kdc[3760](info): commencing operation
krb5kdc: starting...
W20260806 18:37:10.176183 20232 mini_kdc.cc:121] Time spent starting KDC: real 1.032s user 0.001s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786041246094603-20232-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786041246094603-20232-0/krb5kdc/test-user.keytab.
Aug 06 18:37:10 dist-test-slave-1xrh krb5kdc[3760](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041430, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T18:37:10Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T18:37:10Z Disabled control of system clock
WARNING: no policy specified for kudu/127.19.194.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.19.194.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.19.194.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786041246094603-20232-0/krb5kdc/kudu_127.19.194.62.keytab.
Entry for principal kudu/127.19.194.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786041246094603-20232-0/krb5kdc/kudu_127.19.194.62.keytab.
I20260806 18:37:10.298239 20232 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJW69tn/build/debug/bin/kudu
/tmp/dist-test-taskJW69tn/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786041246094603-20232-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786041246094603-20232-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786041246094603-20232-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786041246094603-20232-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.19.194.62:36421
--webserver_interface=127.19.194.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786041246094603-20232-0/krb5kdc/kudu_127.19.194.62.keytab
--principal=kudu/127.19.194.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.19.194.20:36777
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.19.194.62:36421
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20 with env {KRB5CCNAME=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786041246094603-20232-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786041246094603-20232-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786041246094603-20232-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 18:37:10.406203 3776 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 18:37:10.406487 3776 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 18:37:10.406569 3776 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 18:37:10.411144 3776 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260806 18:37:10.411227 3776 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260806 18:37:10.411270 3776 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260806 18:37:10.411306 3776 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 18:37:10.411348 3776 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 18:37:10.411388 3776 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 18:37:10.411423 3776 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 18:37:10.411458 3776 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 18:37:10.416376 3776 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.194.20:36777
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786041246094603-20232-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786041246094603-20232-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.19.194.62:36421
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786041246094603-20232-0/krb5kdc/kudu_127.19.194.62.keytab
--principal=kudu/127.19.194.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.19.194.62:36421
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786041246094603-20232-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.19.194.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3776
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786041246094603-20232-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 17:59:13 UTC on ccycloud.quasar-wvsnzx.root.comops.site
build id 4571028
I20260806 18:37:10.417390 3776 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 18:37:10.418365 3776 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 18:37:10.426745 3782 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:37:10.426744 3784 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:37:10.426761 3781 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 18:37:10.426775 3776 server_base.cc:1034] running on GCE node
I20260806 18:37:10.427640 3776 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 18:37:10.428722 3776 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 18:37:10.429924 3776 hybrid_clock.cc:648] HybridClock initialized: now 1786041430429892 us; error 46 us; skew 500 ppm
Aug 06 18:37:10 dist-test-slave-1xrh krb5kdc[3760](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041430, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.194.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 18:37:10.432533 3776 init.cc:377] Logged in from keytab as kudu/127.19.194.62@KRBTEST.COM (short username kudu)
I20260806 18:37:10.433454 3776 webserver.cc:460] Webserver started at http://127.19.194.62:39513/ using document root <none> and password file <none>
I20260806 18:37:10.434034 3776 fs_manager.cc:359] Metadata directory not provided
I20260806 18:37:10.434123 3776 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 18:37:10.434347 3776 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 18:37:10.436401 3776 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786041246094603-20232-0/minicluster-data/master-0/data/instance:
uuid: "34dfff54533849c68cf3a4743721aeb5"
format_stamp: "Formatted at 2026-08-06 18:37:10 on dist-test-slave-1xrh"
server_key: "6ac8ff971ade94db00b2ef63baf3c45d"
server_key_iv: "c0bad579135a7450618b2f3d295e550d"
server_key_version: "encryptionkey@0"
I20260806 18:37:10.436966 3776 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786041246094603-20232-0/minicluster-data/master-0/wal/instance:
uuid: "34dfff54533849c68cf3a4743721aeb5"
format_stamp: "Formatted at 2026-08-06 18:37:10 on dist-test-slave-1xrh"
server_key: "6ac8ff971ade94db00b2ef63baf3c45d"
server_key_iv: "c0bad579135a7450618b2f3d295e550d"
server_key_version: "encryptionkey@0"
I20260806 18:37:10.440443 3776 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20260806 18:37:10.442760 3791 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 18:37:10.444044 3776 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260806 18:37:10.444182 3776 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786041246094603-20232-0/minicluster-data/master-0/data,/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786041246094603-20232-0/minicluster-data/master-0/wal
uuid: "34dfff54533849c68cf3a4743721aeb5"
format_stamp: "Formatted at 2026-08-06 18:37:10 on dist-test-slave-1xrh"
server_key: "6ac8ff971ade94db00b2ef63baf3c45d"
server_key_iv: "c0bad579135a7450618b2f3d295e550d"
server_key_version: "encryptionkey@0"
I20260806 18:37:10.444288 3776 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786041246094603-20232-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786041246094603-20232-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786041246094603-20232-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 18:37:10.466871 3776 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 18:37:10.469560 3776 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 18:37:10.469766 3776 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 18:37:10.476763 3776 rpc_server.cc:307] RPC server started. Bound to: 127.19.194.62:36421
I20260806 18:37:10.477072 3845 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.194.62:36421 every 8 connection(s)
I20260806 18:37:10.477706 3776 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786041246094603-20232-0/minicluster-data/master-0/data/info.pb
I20260806 18:37:10.480418 3846 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 18:37:10.483907 20232 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJW69tn/build/debug/bin/kudu as pid 3776
I20260806 18:37:10.484037 20232 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786041246094603-20232-0/minicluster-data/master-0/wal/instance
I20260806 18:37:10.484340 20232 external_mini_cluster.cc:1428] Setting key 40e2d5bd30f4bef12a98c54990d9ee77
I20260806 18:37:10.486311 3846 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 34dfff54533849c68cf3a4743721aeb5: Bootstrap starting.
I20260806 18:37:10.488801 3846 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 34dfff54533849c68cf3a4743721aeb5: Neither blocks nor log segments found. Creating new log.
I20260806 18:37:10.489558 3846 log.cc:824] T 00000000000000000000000000000000 P 34dfff54533849c68cf3a4743721aeb5: Log is configured to *not* fsync() on all Append() calls
I20260806 18:37:10.491683 3846 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 34dfff54533849c68cf3a4743721aeb5: No bootstrap required, opened a new log
Aug 06 18:37:10 dist-test-slave-1xrh krb5kdc[3760](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041430, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.19.194.62@KRBTEST.COM
I20260806 18:37:10.495501 3846 raft_consensus.cc:348] T 00000000000000000000000000000000 P 34dfff54533849c68cf3a4743721aeb5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "34dfff54533849c68cf3a4743721aeb5" member_type: VOTER last_known_addr { host: "127.19.194.62" port: 36421 } }
I20260806 18:37:10.495718 3846 raft_consensus.cc:374] T 00000000000000000000000000000000 P 34dfff54533849c68cf3a4743721aeb5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 18:37:10.495779 3846 raft_consensus.cc:729] T 00000000000000000000000000000000 P 34dfff54533849c68cf3a4743721aeb5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 34dfff54533849c68cf3a4743721aeb5, State: Initialized, Role: FOLLOWER
I20260806 18:37:10.496201 3846 consensus_queue.cc:260] T 00000000000000000000000000000000 P 34dfff54533849c68cf3a4743721aeb5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "34dfff54533849c68cf3a4743721aeb5" member_type: VOTER last_known_addr { host: "127.19.194.62" port: 36421 } }
I20260806 18:37:10.496357 3846 raft_consensus.cc:388] T 00000000000000000000000000000000 P 34dfff54533849c68cf3a4743721aeb5 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 18:37:10.496466 3846 raft_consensus.cc:482] T 00000000000000000000000000000000 P 34dfff54533849c68cf3a4743721aeb5 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 18:37:10.496598 3846 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 34dfff54533849c68cf3a4743721aeb5 [term 0 FOLLOWER]: Advancing to term 1
I20260806 18:37:10.497450 3846 raft_consensus.cc:504] T 00000000000000000000000000000000 P 34dfff54533849c68cf3a4743721aeb5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "34dfff54533849c68cf3a4743721aeb5" member_type: VOTER last_known_addr { host: "127.19.194.62" port: 36421 } }
I20260806 18:37:10.497793 3846 leader_election.cc:302] T 00000000000000000000000000000000 P 34dfff54533849c68cf3a4743721aeb5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 34dfff54533849c68cf3a4743721aeb5; no voters:
I20260806 18:37:10.498180 3846 leader_election.cc:288] T 00000000000000000000000000000000 P 34dfff54533849c68cf3a4743721aeb5 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 18:37:10.498255 3851 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 34dfff54533849c68cf3a4743721aeb5 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 18:37:10.498458 3851 raft_consensus.cc:686] T 00000000000000000000000000000000 P 34dfff54533849c68cf3a4743721aeb5 [term 1 LEADER]: Becoming Leader. State: Replica: 34dfff54533849c68cf3a4743721aeb5, State: Running, Role: LEADER
I20260806 18:37:10.498822 3851 consensus_queue.cc:237] T 00000000000000000000000000000000 P 34dfff54533849c68cf3a4743721aeb5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "34dfff54533849c68cf3a4743721aeb5" member_type: VOTER last_known_addr { host: "127.19.194.62" port: 36421 } }
I20260806 18:37:10.499048 3849 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:37:10.485709 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44470 (local address 127.19.194.62:36421)
0806 18:37:10.486117 (+ 408us) server_negotiation.cc:206] Beginning negotiation
0806 18:37:10.486134 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0806 18:37:10.486168 (+ 34us) server_negotiation.cc:407] Connection header received
0806 18:37:10.486856 (+ 688us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:37:10.486872 (+ 16us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:37:10.487243 (+ 371us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:37:10.487511 (+ 268us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:37:10.488404 (+ 893us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:37:10.489441 (+ 1037us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:37:10.490514 (+ 1073us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:37:10.490997 (+ 483us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:37:10.491115 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:37:10.493923 (+ 2808us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:37:10.493929 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:37:10.493941 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 18:37:10.493967 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:37:10.495810 (+ 1843us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:37:10.496320 (+ 510us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:37:10.496326 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:37:10.496332 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:37:10.496528 (+ 196us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:37:10.496857 (+ 329us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:37:10.496861 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:37:10.496863 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:37:10.497239 (+ 376us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:37:10.497465 (+ 226us) server_negotiation.cc:1035] Waiting for connection context
0806 18:37:10.497731 (+ 266us) server_negotiation.cc:299] Negotiation successful
0806 18:37:10.498058 (+ 327us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":264,"thread_start_us":141,"threads_started":1}
I20260806 18:37:10.499472 3846 sys_catalog.cc:564] T 00000000000000000000000000000000 P 34dfff54533849c68cf3a4743721aeb5 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 18:37:10.500037 3851 sys_catalog.cc:455] T 00000000000000000000000000000000 P 34dfff54533849c68cf3a4743721aeb5 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 34dfff54533849c68cf3a4743721aeb5. Latest consensus state: current_term: 1 leader_uuid: "34dfff54533849c68cf3a4743721aeb5" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "34dfff54533849c68cf3a4743721aeb5" member_type: VOTER last_known_addr { host: "127.19.194.62" port: 36421 } } }
I20260806 18:37:10.500247 3851 sys_catalog.cc:458] T 00000000000000000000000000000000 P 34dfff54533849c68cf3a4743721aeb5 [sys.catalog]: This master's current role is: LEADER
I20260806 18:37:10.500483 3852 sys_catalog.cc:455] T 00000000000000000000000000000000 P 34dfff54533849c68cf3a4743721aeb5 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "34dfff54533849c68cf3a4743721aeb5" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "34dfff54533849c68cf3a4743721aeb5" member_type: VOTER last_known_addr { host: "127.19.194.62" port: 36421 } } }
I20260806 18:37:10.500648 3852 sys_catalog.cc:458] T 00000000000000000000000000000000 P 34dfff54533849c68cf3a4743721aeb5 [sys.catalog]: This master's current role is: LEADER
I20260806 18:37:10.501343 3859 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 18:37:10.504477 3859 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 18:37:10.510179 3859 catalog_manager.cc:1347] Generated new cluster ID: 9dc90a31f80745688367d0fd1a89a281
I20260806 18:37:10.510255 3859 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 18:37:10.524426 3859 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 18:37:10.525260 3859 catalog_manager.cc:1504] Loading token signing keys...
I20260806 18:37:10.530037 3859 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 34dfff54533849c68cf3a4743721aeb5: Generated new TSK 0
I20260806 18:37:10.530619 3859 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.19.194.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.19.194.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.19.194.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786041246094603-20232-0/krb5kdc/kudu_127.19.194.1.keytab.
Entry for principal kudu/127.19.194.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786041246094603-20232-0/krb5kdc/kudu_127.19.194.1.keytab.
I20260806 18:37:10.562489 20232 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJW69tn/build/debug/bin/kudu
/tmp/dist-test-taskJW69tn/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786041246094603-20232-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786041246094603-20232-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786041246094603-20232-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786041246094603-20232-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.19.194.1:0
--local_ip_for_outbound_sockets=127.19.194.1
--webserver_interface=127.19.194.1
--webserver_port=0
--tserver_master_addrs=127.19.194.62:36421
--keytab_file=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786041246094603-20232-0/krb5kdc/kudu_127.19.194.1.keytab
--principal=kudu/127.19.194.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.19.194.20:36777
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786041246094603-20232-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786041246094603-20232-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786041246094603-20232-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 18:37:10.700093 3873 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 18:37:10.700376 3873 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 18:37:10.700450 3873 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 18:37:10.704365 3873 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 18:37:10.704416 3873 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 18:37:10.704473 3873 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.19.194.1
I20260806 18:37:10.709584 3873 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.194.20:36777
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786041246094603-20232-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786041246094603-20232-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786041246094603-20232-0/krb5kdc/kudu_127.19.194.1.keytab
--principal=kudu/127.19.194.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.19.194.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786041246094603-20232-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.19.194.1
--webserver_port=0
--tserver_master_addrs=127.19.194.62:36421
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3873
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.19.194.1
--log_dir=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786041246094603-20232-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 17:59:13 UTC on ccycloud.quasar-wvsnzx.root.comops.site
build id 4571028
I20260806 18:37:10.710458 3873 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 18:37:10.711372 3873 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 18:37:10.720737 3881 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:37:10.720813 3879 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:37:10.720808 3878 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 18:37:10.721570 3873 server_base.cc:1034] running on GCE node
I20260806 18:37:10.721937 3873 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 18:37:10.722478 3873 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 18:37:10.723660 3873 hybrid_clock.cc:648] HybridClock initialized: now 1786041430723641 us; error 29 us; skew 500 ppm
Aug 06 18:37:10 dist-test-slave-1xrh krb5kdc[3760](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041430, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.194.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 18:37:10.726321 3873 init.cc:377] Logged in from keytab as kudu/127.19.194.1@KRBTEST.COM (short username kudu)
I20260806 18:37:10.727159 3873 webserver.cc:460] Webserver started at http://127.19.194.1:40163/ using document root <none> and password file <none>
I20260806 18:37:10.727732 3873 fs_manager.cc:359] Metadata directory not provided
I20260806 18:37:10.727828 3873 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 18:37:10.728061 3873 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 18:37:10.730054 3873 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786041246094603-20232-0/minicluster-data/ts-0/data/instance:
uuid: "5fdb1c79f09d42078b9fadc4624e6d0d"
format_stamp: "Formatted at 2026-08-06 18:37:10 on dist-test-slave-1xrh"
server_key: "18754be74733259eab711d3b24a590eb"
server_key_iv: "532b38fc980d1832027c7f020488d752"
server_key_version: "encryptionkey@0"
I20260806 18:37:10.730644 3873 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786041246094603-20232-0/minicluster-data/ts-0/wal/instance:
uuid: "5fdb1c79f09d42078b9fadc4624e6d0d"
format_stamp: "Formatted at 2026-08-06 18:37:10 on dist-test-slave-1xrh"
server_key: "18754be74733259eab711d3b24a590eb"
server_key_iv: "532b38fc980d1832027c7f020488d752"
server_key_version: "encryptionkey@0"
I20260806 18:37:10.734436 3873 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260806 18:37:10.736855 3889 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 18:37:10.738020 3873 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.000s sys 0.001s
I20260806 18:37:10.738176 3873 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786041246094603-20232-0/minicluster-data/ts-0/data,/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786041246094603-20232-0/minicluster-data/ts-0/wal
uuid: "5fdb1c79f09d42078b9fadc4624e6d0d"
format_stamp: "Formatted at 2026-08-06 18:37:10 on dist-test-slave-1xrh"
server_key: "18754be74733259eab711d3b24a590eb"
server_key_iv: "532b38fc980d1832027c7f020488d752"
server_key_version: "encryptionkey@0"
I20260806 18:37:10.738287 3873 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786041246094603-20232-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786041246094603-20232-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786041246094603-20232-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 18:37:10.759572 3873 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 18:37:10.762224 3873 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 18:37:10.762416 3873 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 18:37:10.762951 3873 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 18:37:10.763931 3873 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 18:37:10.763993 3873 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 18:37:10.764068 3873 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 18:37:10.764098 3873 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 18:37:10.773376 3873 rpc_server.cc:307] RPC server started. Bound to: 127.19.194.1:36577
I20260806 18:37:10.773401 4007 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.194.1:36577 every 8 connection(s)
I20260806 18:37:10.774179 3873 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786041246094603-20232-0/minicluster-data/ts-0/data/info.pb
I20260806 18:37:10.779441 20232 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJW69tn/build/debug/bin/kudu as pid 3873
I20260806 18:37:10.779572 20232 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786041246094603-20232-0/minicluster-data/ts-0/wal/instance
I20260806 18:37:10.779929 20232 external_mini_cluster.cc:1428] Setting key 325f61cd6d190fb4815b37110e8fbac1
Aug 06 18:37:10 dist-test-slave-1xrh krb5kdc[3760](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041430, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.194.1@KRBTEST.COM for kudu/127.19.194.62@KRBTEST.COM
I20260806 18:37:10.792429 3849 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:37:10.775995 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.194.1:50731 (local address 127.19.194.62:36421)
0806 18:37:10.776126 (+ 131us) server_negotiation.cc:206] Beginning negotiation
0806 18:37:10.776130 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 18:37:10.776919 (+ 789us) server_negotiation.cc:407] Connection header received
0806 18:37:10.778627 (+ 1708us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:37:10.778630 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:37:10.778692 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:37:10.778779 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:37:10.780483 (+ 1704us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:37:10.781431 (+ 948us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:37:10.783267 (+ 1836us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:37:10.783672 (+ 405us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:37:10.783802 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:37:10.788376 (+ 4574us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:37:10.788380 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:37:10.788382 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 18:37:10.788409 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:37:10.790145 (+ 1736us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:37:10.790687 (+ 542us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:37:10.790692 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:37:10.790696 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:37:10.790795 (+ 99us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:37:10.791178 (+ 383us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:37:10.791184 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:37:10.791186 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:37:10.791441 (+ 255us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:37:10.791561 (+ 120us) server_negotiation.cc:1035] Waiting for connection context
0806 18:37:10.792061 (+ 500us) server_negotiation.cc:299] Negotiation successful
0806 18:37:10.792220 (+ 159us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
WARNING: no policy specified for kudu/127.19.194.2@KRBTEST.COM; defaulting to no policy
I20260806 18:37:10.793155 4010 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:37:10.776289 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.194.62:36421 (local address 127.19.194.1:50731)
0806 18:37:10.776740 (+ 451us) negotiation.cc:107] Waiting for socket to connect
0806 18:37:10.776782 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0806 18:37:10.778368 (+ 1586us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:37:10.778978 (+ 610us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:37:10.778988 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:37:10.779566 (+ 578us) client_negotiation.cc:189] Negotiated authn=SASL
0806 18:37:10.780235 (+ 669us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:37:10.780256 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:37:10.781621 (+ 1365us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:37:10.781626 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:37:10.783096 (+ 1470us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:37:10.783108 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:37:10.785211 (+ 2103us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:37:10.785213 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:37:10.785460 (+ 247us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:37:10.786128 (+ 668us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 18:37:10.786152 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0806 18:37:10.788187 (+ 2035us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 18:37:10.790283 (+ 2096us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:37:10.790289 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:37:10.790302 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:37:10.790563 (+ 261us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:37:10.790910 (+ 347us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:37:10.790913 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:37:10.790914 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:37:10.791043 (+ 129us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:37:10.791548 (+ 505us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 18:37:10.791553 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 18:37:10.791781 (+ 228us) client_negotiation.cc:769] Sending connection context
0806 18:37:10.791938 (+ 157us) client_negotiation.cc:240] Negotiation successful
0806 18:37:10.792190 (+ 252us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":277,"thread_start_us":104,"threads_started":1}
I20260806 18:37:10.794397 4008 heartbeater.cc:347] Connected to a master server at 127.19.194.62:36421
I20260806 18:37:10.794672 4008 heartbeater.cc:464] Registering TS with master...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.19.194.2@KRBTEST.COM" created.
I20260806 18:37:10.795341 4008 heartbeater.cc:511] Master 127.19.194.62:36421 requested a full tablet report, sending...
I20260806 18:37:10.797096 3809 ts_manager.cc:194] Registered new tserver with Master: 5fdb1c79f09d42078b9fadc4624e6d0d (127.19.194.1:36577)
I20260806 18:37:10.798295 3809 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.19.194.1@KRBTEST.COM'} at 127.19.194.1:50731
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.19.194.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786041246094603-20232-0/krb5kdc/kudu_127.19.194.2.keytab.
Entry for principal kudu/127.19.194.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786041246094603-20232-0/krb5kdc/kudu_127.19.194.2.keytab.
I20260806 18:37:10.814898 20232 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJW69tn/build/debug/bin/kudu
/tmp/dist-test-taskJW69tn/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786041246094603-20232-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786041246094603-20232-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786041246094603-20232-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786041246094603-20232-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.19.194.2:0
--local_ip_for_outbound_sockets=127.19.194.2
--webserver_interface=127.19.194.2
--webserver_port=0
--tserver_master_addrs=127.19.194.62:36421
--keytab_file=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786041246094603-20232-0/krb5kdc/kudu_127.19.194.2.keytab
--principal=kudu/127.19.194.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.19.194.20:36777
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786041246094603-20232-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786041246094603-20232-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786041246094603-20232-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 18:37:10.923648 4013 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 18:37:10.923931 4013 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 18:37:10.924007 4013 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 18:37:10.928053 4013 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 18:37:10.928129 4013 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 18:37:10.928216 4013 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.19.194.2
I20260806 18:37:10.933609 4013 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.194.20:36777
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786041246094603-20232-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786041246094603-20232-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786041246094603-20232-0/krb5kdc/kudu_127.19.194.2.keytab
--principal=kudu/127.19.194.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.19.194.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786041246094603-20232-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.19.194.2
--webserver_port=0
--tserver_master_addrs=127.19.194.62:36421
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4013
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.19.194.2
--log_dir=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786041246094603-20232-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 17:59:13 UTC on ccycloud.quasar-wvsnzx.root.comops.site
build id 4571028
I20260806 18:37:10.934579 4013 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 18:37:10.935634 4013 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 18:37:10.945107 4019 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:37:10.945163 4021 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:37:10.945173 4018 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 18:37:10.945206 4013 server_base.cc:1034] running on GCE node
I20260806 18:37:10.945719 4013 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 18:37:10.946319 4013 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 18:37:10.947504 4013 hybrid_clock.cc:648] HybridClock initialized: now 1786041430947491 us; error 26 us; skew 500 ppm
Aug 06 18:37:10 dist-test-slave-1xrh krb5kdc[3760](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041430, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.194.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 18:37:10.950332 4013 init.cc:377] Logged in from keytab as kudu/127.19.194.2@KRBTEST.COM (short username kudu)
I20260806 18:37:10.951207 4013 webserver.cc:460] Webserver started at http://127.19.194.2:38271/ using document root <none> and password file <none>
I20260806 18:37:10.951678 4013 fs_manager.cc:359] Metadata directory not provided
I20260806 18:37:10.951735 4013 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 18:37:10.951884 4013 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 18:37:10.953729 4013 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786041246094603-20232-0/minicluster-data/ts-1/data/instance:
uuid: "4d2a68a0b77a43619f6206aee728280d"
format_stamp: "Formatted at 2026-08-06 18:37:10 on dist-test-slave-1xrh"
server_key: "f9d523d313ee70f29dd5f5475c7ff29c"
server_key_iv: "313833d9c7638513ffd6527402988301"
server_key_version: "encryptionkey@0"
I20260806 18:37:10.954241 4013 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786041246094603-20232-0/minicluster-data/ts-1/wal/instance:
uuid: "4d2a68a0b77a43619f6206aee728280d"
format_stamp: "Formatted at 2026-08-06 18:37:10 on dist-test-slave-1xrh"
server_key: "f9d523d313ee70f29dd5f5475c7ff29c"
server_key_iv: "313833d9c7638513ffd6527402988301"
server_key_version: "encryptionkey@0"
I20260806 18:37:10.957777 4013 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.006s sys 0.000s
I20260806 18:37:10.960014 4028 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 18:37:10.961239 4013 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260806 18:37:10.961369 4013 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786041246094603-20232-0/minicluster-data/ts-1/data,/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786041246094603-20232-0/minicluster-data/ts-1/wal
uuid: "4d2a68a0b77a43619f6206aee728280d"
format_stamp: "Formatted at 2026-08-06 18:37:10 on dist-test-slave-1xrh"
server_key: "f9d523d313ee70f29dd5f5475c7ff29c"
server_key_iv: "313833d9c7638513ffd6527402988301"
server_key_version: "encryptionkey@0"
I20260806 18:37:10.961469 4013 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786041246094603-20232-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786041246094603-20232-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786041246094603-20232-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 18:37:10.984402 4013 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 18:37:10.987294 4013 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 18:37:10.987493 4013 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 18:37:10.988083 4013 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 18:37:10.988899 4013 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 18:37:10.988947 4013 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 18:37:10.989023 4013 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 18:37:10.989061 4013 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 18:37:10.997814 4013 rpc_server.cc:307] RPC server started. Bound to: 127.19.194.2:46303
I20260806 18:37:10.997840 4148 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.194.2:46303 every 8 connection(s)
I20260806 18:37:10.999012 4013 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786041246094603-20232-0/minicluster-data/ts-1/data/info.pb
I20260806 18:37:11.000937 20232 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJW69tn/build/debug/bin/kudu as pid 4013
I20260806 18:37:11.001081 20232 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786041246094603-20232-0/minicluster-data/ts-1/wal/instance
I20260806 18:37:11.001430 20232 external_mini_cluster.cc:1428] Setting key d3ff09f939c45ad8b7ffdf6d7655d8b6
Aug 06 18:37:11 dist-test-slave-1xrh krb5kdc[3760](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041430, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.194.2@KRBTEST.COM for kudu/127.19.194.62@KRBTEST.COM
WARNING: no policy specified for kudu/127.19.194.3@KRBTEST.COM; defaulting to no policy
I20260806 18:37:11.013718 3849 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:37:11.001081 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.194.2:52727 (local address 127.19.194.62:36421)
0806 18:37:11.001227 (+ 146us) server_negotiation.cc:206] Beginning negotiation
0806 18:37:11.001231 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 18:37:11.002132 (+ 901us) server_negotiation.cc:407] Connection header received
0806 18:37:11.003827 (+ 1695us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:37:11.003829 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:37:11.003890 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:37:11.003959 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:37:11.005534 (+ 1575us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:37:11.006056 (+ 522us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:37:11.007236 (+ 1180us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:37:11.007466 (+ 230us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:37:11.007589 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:37:11.010109 (+ 2520us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:37:11.010114 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:37:11.010118 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 18:37:11.010148 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:37:11.011790 (+ 1642us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:37:11.012313 (+ 523us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:37:11.012316 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:37:11.012317 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:37:11.012391 (+ 74us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:37:11.012748 (+ 357us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:37:11.012754 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:37:11.012758 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:37:11.012954 (+ 196us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:37:11.013043 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0806 18:37:11.013488 (+ 445us) server_negotiation.cc:299] Negotiation successful
0806 18:37:11.013585 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260806 18:37:11.014513 4151 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:37:11.001487 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.194.62:36421 (local address 127.19.194.2:52727)
0806 18:37:11.001984 (+ 497us) negotiation.cc:107] Waiting for socket to connect
0806 18:37:11.002033 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0806 18:37:11.003595 (+ 1562us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:37:11.004147 (+ 552us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:37:11.004159 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:37:11.004676 (+ 517us) client_negotiation.cc:189] Negotiated authn=SASL
0806 18:37:11.005372 (+ 696us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:37:11.005388 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:37:11.006187 (+ 799us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:37:11.006190 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:37:11.007065 (+ 875us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:37:11.007075 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:37:11.007626 (+ 551us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:37:11.007631 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:37:11.007800 (+ 169us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:37:11.008468 (+ 668us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 18:37:11.008491 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0806 18:37:11.009935 (+ 1444us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 18:37:11.011957 (+ 2022us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:37:11.011961 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:37:11.011972 (+ 11us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:37:11.012215 (+ 243us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:37:11.012476 (+ 261us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:37:11.012478 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:37:11.012479 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:37:11.012614 (+ 135us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:37:11.013062 (+ 448us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 18:37:11.013066 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 18:37:11.013285 (+ 219us) client_negotiation.cc:769] Sending connection context
0806 18:37:11.013433 (+ 148us) client_negotiation.cc:240] Negotiation successful
0806 18:37:11.013692 (+ 259us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":288,"thread_start_us":108,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.19.194.3@KRBTEST.COM" created.
I20260806 18:37:11.015791 4149 heartbeater.cc:347] Connected to a master server at 127.19.194.62:36421
I20260806 18:37:11.016029 4149 heartbeater.cc:464] Registering TS with master...
I20260806 18:37:11.016587 4149 heartbeater.cc:511] Master 127.19.194.62:36421 requested a full tablet report, sending...
I20260806 18:37:11.017763 3809 ts_manager.cc:194] Registered new tserver with Master: 4d2a68a0b77a43619f6206aee728280d (127.19.194.2:46303)
I20260806 18:37:11.018338 3809 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.19.194.2@KRBTEST.COM'} at 127.19.194.2:52727
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.19.194.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786041246094603-20232-0/krb5kdc/kudu_127.19.194.3.keytab.
Entry for principal kudu/127.19.194.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786041246094603-20232-0/krb5kdc/kudu_127.19.194.3.keytab.
I20260806 18:37:11.030671 20232 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJW69tn/build/debug/bin/kudu
/tmp/dist-test-taskJW69tn/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786041246094603-20232-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786041246094603-20232-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786041246094603-20232-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786041246094603-20232-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.19.194.3:0
--local_ip_for_outbound_sockets=127.19.194.3
--webserver_interface=127.19.194.3
--webserver_port=0
--tserver_master_addrs=127.19.194.62:36421
--keytab_file=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786041246094603-20232-0/krb5kdc/kudu_127.19.194.3.keytab
--principal=kudu/127.19.194.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.19.194.20:36777
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786041246094603-20232-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786041246094603-20232-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786041246094603-20232-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 18:37:11.139971 4154 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 18:37:11.140196 4154 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 18:37:11.140266 4154 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 18:37:11.144385 4154 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 18:37:11.144436 4154 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 18:37:11.144541 4154 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.19.194.3
I20260806 18:37:11.150103 4154 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.194.20:36777
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786041246094603-20232-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786041246094603-20232-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786041246094603-20232-0/krb5kdc/kudu_127.19.194.3.keytab
--principal=kudu/127.19.194.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.19.194.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786041246094603-20232-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.19.194.3
--webserver_port=0
--tserver_master_addrs=127.19.194.62:36421
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4154
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.19.194.3
--log_dir=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786041246094603-20232-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 17:59:13 UTC on ccycloud.quasar-wvsnzx.root.comops.site
build id 4571028
I20260806 18:37:11.151051 4154 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 18:37:11.152029 4154 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 18:37:11.161326 4162 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:37:11.161326 4160 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 18:37:11.161412 4154 server_base.cc:1034] running on GCE node
W20260806 18:37:11.161331 4159 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 18:37:11.161912 4154 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 18:37:11.162520 4154 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 18:37:11.163698 4154 hybrid_clock.cc:648] HybridClock initialized: now 1786041431163690 us; error 28 us; skew 500 ppm
Aug 06 18:37:11 dist-test-slave-1xrh krb5kdc[3760](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041431, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.194.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 18:37:11.166445 4154 init.cc:377] Logged in from keytab as kudu/127.19.194.3@KRBTEST.COM (short username kudu)
I20260806 18:37:11.167338 4154 webserver.cc:460] Webserver started at http://127.19.194.3:33259/ using document root <none> and password file <none>
I20260806 18:37:11.167941 4154 fs_manager.cc:359] Metadata directory not provided
I20260806 18:37:11.168035 4154 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 18:37:11.168264 4154 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 18:37:11.170185 4154 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786041246094603-20232-0/minicluster-data/ts-2/data/instance:
uuid: "c5a13411dadf4739ba09a5da11f16926"
format_stamp: "Formatted at 2026-08-06 18:37:11 on dist-test-slave-1xrh"
server_key: "702a906614a2f7717547b4cb83b505a1"
server_key_iv: "8fafdc8e52d92aa949a83dcb793bd92c"
server_key_version: "encryptionkey@0"
I20260806 18:37:11.170742 4154 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786041246094603-20232-0/minicluster-data/ts-2/wal/instance:
uuid: "c5a13411dadf4739ba09a5da11f16926"
format_stamp: "Formatted at 2026-08-06 18:37:11 on dist-test-slave-1xrh"
server_key: "702a906614a2f7717547b4cb83b505a1"
server_key_iv: "8fafdc8e52d92aa949a83dcb793bd92c"
server_key_version: "encryptionkey@0"
I20260806 18:37:11.174387 4154 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.002s sys 0.004s
I20260806 18:37:11.176674 4169 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 18:37:11.177814 4154 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260806 18:37:11.177975 4154 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786041246094603-20232-0/minicluster-data/ts-2/data,/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786041246094603-20232-0/minicluster-data/ts-2/wal
uuid: "c5a13411dadf4739ba09a5da11f16926"
format_stamp: "Formatted at 2026-08-06 18:37:11 on dist-test-slave-1xrh"
server_key: "702a906614a2f7717547b4cb83b505a1"
server_key_iv: "8fafdc8e52d92aa949a83dcb793bd92c"
server_key_version: "encryptionkey@0"
I20260806 18:37:11.178083 4154 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786041246094603-20232-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786041246094603-20232-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786041246094603-20232-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 18:37:11.195652 4154 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 18:37:11.198460 4154 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 18:37:11.198653 4154 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 18:37:11.199292 4154 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 18:37:11.200103 4154 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 18:37:11.200170 4154 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 18:37:11.200240 4154 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 18:37:11.200294 4154 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 18:37:11.209224 4154 rpc_server.cc:307] RPC server started. Bound to: 127.19.194.3:35207
I20260806 18:37:11.209257 4292 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.194.3:35207 every 8 connection(s)
I20260806 18:37:11.210235 4154 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786041246094603-20232-0/minicluster-data/ts-2/data/info.pb
I20260806 18:37:11.217097 20232 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJW69tn/build/debug/bin/kudu as pid 4154
I20260806 18:37:11.217229 20232 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786041246094603-20232-0/minicluster-data/ts-2/wal/instance
I20260806 18:37:11.217550 20232 external_mini_cluster.cc:1428] Setting key 5a00ba4c3e88dd5b5f6d9ee1a99f2f8b
Aug 06 18:37:11 dist-test-slave-1xrh krb5kdc[3760](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041431, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.194.3@KRBTEST.COM for kudu/127.19.194.62@KRBTEST.COM
I20260806 18:37:11.224022 3849 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:37:11.212074 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.194.3:57449 (local address 127.19.194.62:36421)
0806 18:37:11.212219 (+ 145us) server_negotiation.cc:206] Beginning negotiation
0806 18:37:11.212222 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 18:37:11.213007 (+ 785us) server_negotiation.cc:407] Connection header received
0806 18:37:11.214349 (+ 1342us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:37:11.214352 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:37:11.214409 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:37:11.214514 (+ 105us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:37:11.215661 (+ 1147us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:37:11.216183 (+ 522us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:37:11.217670 (+ 1487us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:37:11.217902 (+ 232us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:37:11.218005 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:37:11.220724 (+ 2719us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:37:11.220727 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:37:11.220729 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 18:37:11.220754 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:37:11.221963 (+ 1209us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:37:11.222490 (+ 527us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:37:11.222499 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:37:11.222500 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:37:11.222574 (+ 74us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:37:11.222911 (+ 337us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:37:11.222914 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:37:11.222916 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:37:11.223090 (+ 174us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:37:11.223245 (+ 155us) server_negotiation.cc:1035] Waiting for connection context
0806 18:37:11.223710 (+ 465us) server_negotiation.cc:299] Negotiation successful
0806 18:37:11.223854 (+ 144us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260806 18:37:11.224766 4295 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:37:11.212444 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.194.62:36421 (local address 127.19.194.3:57449)
0806 18:37:11.212860 (+ 416us) negotiation.cc:107] Waiting for socket to connect
0806 18:37:11.212901 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0806 18:37:11.214137 (+ 1236us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:37:11.214691 (+ 554us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:37:11.214698 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:37:11.215045 (+ 347us) client_negotiation.cc:189] Negotiated authn=SASL
0806 18:37:11.215499 (+ 454us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:37:11.215512 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:37:11.216315 (+ 803us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:37:11.216318 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:37:11.217507 (+ 1189us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:37:11.217521 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:37:11.218086 (+ 565us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:37:11.218091 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:37:11.218276 (+ 185us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:37:11.218873 (+ 597us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 18:37:11.218888 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0806 18:37:11.220561 (+ 1673us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 18:37:11.222084 (+ 1523us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:37:11.222090 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:37:11.222103 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:37:11.222362 (+ 259us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:37:11.222665 (+ 303us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:37:11.222668 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:37:11.222670 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:37:11.222800 (+ 130us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:37:11.223245 (+ 445us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 18:37:11.223252 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 18:37:11.223480 (+ 228us) client_negotiation.cc:769] Sending connection context
0806 18:37:11.223666 (+ 186us) client_negotiation.cc:240] Negotiation successful
0806 18:37:11.223994 (+ 328us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":246,"thread_start_us":108,"threads_started":1}
I20260806 18:37:11.225782 4293 heartbeater.cc:347] Connected to a master server at 127.19.194.62:36421
I20260806 18:37:11.226018 4293 heartbeater.cc:464] Registering TS with master...
I20260806 18:37:11.226488 4293 heartbeater.cc:511] Master 127.19.194.62:36421 requested a full tablet report, sending...
I20260806 18:37:11.227505 3809 ts_manager.cc:194] Registered new tserver with Master: c5a13411dadf4739ba09a5da11f16926 (127.19.194.3:35207)
I20260806 18:37:11.228086 3809 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.19.194.3@KRBTEST.COM'} at 127.19.194.3:57449
I20260806 18:37:11.231504 20232 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260806 18:37:11.240931 3849 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:37:11.233441 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44482 (local address 127.19.194.62:36421)
0806 18:37:11.233552 (+ 111us) server_negotiation.cc:206] Beginning negotiation
0806 18:37:11.233556 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 18:37:11.233785 (+ 229us) server_negotiation.cc:407] Connection header received
0806 18:37:11.233881 (+ 96us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:37:11.233883 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:37:11.233918 (+ 35us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:37:11.233979 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:37:11.234651 (+ 672us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:37:11.235158 (+ 507us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:37:11.236000 (+ 842us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:37:11.236211 (+ 211us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:37:11.236298 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:37:11.237511 (+ 1213us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:37:11.237514 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:37:11.237515 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 18:37:11.237536 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:37:11.238959 (+ 1423us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:37:11.239494 (+ 535us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:37:11.239498 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:37:11.239500 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:37:11.239633 (+ 133us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:37:11.239966 (+ 333us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:37:11.239972 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:37:11.239976 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:37:11.240305 (+ 329us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:37:11.240439 (+ 134us) server_negotiation.cc:1035] Waiting for connection context
0806 18:37:11.240709 (+ 270us) server_negotiation.cc:299] Negotiation successful
0806 18:37:11.240795 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260806 18:37:11.243705 3809 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:44482:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260806 18:37:11.246723 3809 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 18:37:11.261161 4308 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:37:11.255266 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51352 (local address 127.19.194.3:35207)
0806 18:37:11.256254 (+ 988us) server_negotiation.cc:206] Beginning negotiation
0806 18:37:11.256264 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0806 18:37:11.256284 (+ 20us) server_negotiation.cc:407] Connection header received
0806 18:37:11.256344 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:37:11.256347 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:37:11.256557 (+ 210us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:37:11.256751 (+ 194us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 18:37:11.257048 (+ 297us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:37:11.258189 (+ 1141us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:37:11.259686 (+ 1497us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:37:11.260485 (+ 799us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:37:11.260601 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:37:11.260741 (+ 140us) server_negotiation.cc:1035] Waiting for connection context
0806 18:37:11.260929 (+ 188us) server_negotiation.cc:299] Negotiation successful
0806 18:37:11.261013 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":882,"thread_start_us":168,"threads_started":1}
I20260806 18:37:11.261150 4309 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:37:11.255069 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.194.3:35207 (local address 127.0.0.1:51352)
0806 18:37:11.255872 (+ 803us) negotiation.cc:107] Waiting for socket to connect
0806 18:37:11.255918 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0806 18:37:11.256045 (+ 127us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:37:11.256760 (+ 715us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:37:11.256767 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:37:11.256791 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 18:37:11.256937 (+ 146us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:37:11.256944 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:37:11.258341 (+ 1397us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:37:11.258344 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:37:11.259544 (+ 1200us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:37:11.259555 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:37:11.260601 (+ 1046us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:37:11.260605 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:37:11.260719 (+ 114us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:37:11.260758 (+ 39us) client_negotiation.cc:769] Sending connection context
0806 18:37:11.260893 (+ 135us) client_negotiation.cc:240] Negotiation successful
0806 18:37:11.261011 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":625,"mutex_wait_us":54,"thread_start_us":82,"threads_started":1}
I20260806 18:37:11.261339 4310 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:37:11.255772 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.194.2:46303 (local address 127.0.0.1:45146)
0806 18:37:11.256853 (+ 1081us) negotiation.cc:107] Waiting for socket to connect
0806 18:37:11.256866 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0806 18:37:11.256954 (+ 88us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:37:11.257400 (+ 446us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:37:11.257403 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:37:11.257420 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 18:37:11.257507 (+ 87us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:37:11.257513 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:37:11.258495 (+ 982us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:37:11.258499 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:37:11.259819 (+ 1320us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:37:11.259829 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:37:11.260917 (+ 1088us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:37:11.260920 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:37:11.261009 (+ 89us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:37:11.261033 (+ 24us) client_negotiation.cc:769] Sending connection context
0806 18:37:11.261107 (+ 74us) client_negotiation.cc:240] Negotiation successful
0806 18:37:11.261178 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1028,"thread_start_us":87,"threads_started":1}
I20260806 18:37:11.261670 4306 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:37:11.255387 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.194.1:36577 (local address 127.0.0.1:46596)
0806 18:37:11.255929 (+ 542us) negotiation.cc:107] Waiting for socket to connect
0806 18:37:11.255944 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0806 18:37:11.256049 (+ 105us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:37:11.257536 (+ 1487us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:37:11.257539 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:37:11.257557 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 18:37:11.257644 (+ 87us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:37:11.257650 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:37:11.259219 (+ 1569us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:37:11.259223 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:37:11.260275 (+ 1052us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:37:11.260287 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:37:11.261313 (+ 1026us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:37:11.261316 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:37:11.261417 (+ 101us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:37:11.261443 (+ 26us) client_negotiation.cc:769] Sending connection context
0806 18:37:11.261497 (+ 54us) client_negotiation.cc:240] Negotiation successful
0806 18:37:11.261560 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":475,"thread_start_us":82,"threads_started":1}
I20260806 18:37:11.261782 4312 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:37:11.256297 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45146 (local address 127.19.194.2:46303)
0806 18:37:11.256669 (+ 372us) server_negotiation.cc:206] Beginning negotiation
0806 18:37:11.256684 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0806 18:37:11.257049 (+ 365us) server_negotiation.cc:407] Connection header received
0806 18:37:11.257127 (+ 78us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:37:11.257131 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:37:11.257270 (+ 139us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:37:11.257422 (+ 152us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 18:37:11.257595 (+ 173us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:37:11.258350 (+ 755us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:37:11.259947 (+ 1597us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:37:11.260625 (+ 678us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:37:11.260727 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:37:11.260854 (+ 127us) server_negotiation.cc:1035] Waiting for connection context
0806 18:37:11.261533 (+ 679us) server_negotiation.cc:299] Negotiation successful
0806 18:37:11.261630 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":226,"thread_start_us":110,"threads_started":1}
I20260806 18:37:11.262398 4311 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:37:11.255977 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46596 (local address 127.19.194.1:36577)
0806 18:37:11.257065 (+ 1088us) server_negotiation.cc:206] Beginning negotiation
0806 18:37:11.257081 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0806 18:37:11.257106 (+ 25us) server_negotiation.cc:407] Connection header received
0806 18:37:11.257179 (+ 73us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:37:11.257184 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:37:11.257364 (+ 180us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:37:11.257793 (+ 429us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 18:37:11.257942 (+ 149us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:37:11.259025 (+ 1083us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:37:11.260412 (+ 1387us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:37:11.261187 (+ 775us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:37:11.261959 (+ 772us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:37:11.262117 (+ 158us) server_negotiation.cc:1035] Waiting for connection context
0806 18:37:11.262165 (+ 48us) server_negotiation.cc:299] Negotiation successful
0806 18:37:11.262251 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":979,"thread_start_us":74,"threads_started":1}
I20260806 18:37:11.264714 4079 tablet_service.cc:1467] Processing CreateTablet for tablet 77eea4fef45b4b2fb4976e06fb15f386 (DEFAULT_TABLE table=test-table [id=cd76f96f476143f6a38dddbaef9c6ea8]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 18:37:11.264704 4219 tablet_service.cc:1467] Processing CreateTablet for tablet 77eea4fef45b4b2fb4976e06fb15f386 (DEFAULT_TABLE table=test-table [id=cd76f96f476143f6a38dddbaef9c6ea8]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 18:37:11.264916 3940 tablet_service.cc:1467] Processing CreateTablet for tablet 77eea4fef45b4b2fb4976e06fb15f386 (DEFAULT_TABLE table=test-table [id=cd76f96f476143f6a38dddbaef9c6ea8]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 18:37:11.265774 3940 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 77eea4fef45b4b2fb4976e06fb15f386. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 18:37:11.265791 4219 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 77eea4fef45b4b2fb4976e06fb15f386. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 18:37:11.265790 4079 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 77eea4fef45b4b2fb4976e06fb15f386. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 18:37:11.271960 4313 tablet_bootstrap.cc:492] T 77eea4fef45b4b2fb4976e06fb15f386 P 4d2a68a0b77a43619f6206aee728280d: Bootstrap starting.
I20260806 18:37:11.273334 4315 tablet_bootstrap.cc:492] T 77eea4fef45b4b2fb4976e06fb15f386 P 5fdb1c79f09d42078b9fadc4624e6d0d: Bootstrap starting.
I20260806 18:37:11.274662 4313 tablet_bootstrap.cc:654] T 77eea4fef45b4b2fb4976e06fb15f386 P 4d2a68a0b77a43619f6206aee728280d: Neither blocks nor log segments found. Creating new log.
I20260806 18:37:11.275061 4315 tablet_bootstrap.cc:654] T 77eea4fef45b4b2fb4976e06fb15f386 P 5fdb1c79f09d42078b9fadc4624e6d0d: Neither blocks nor log segments found. Creating new log.
I20260806 18:37:11.275084 4314 tablet_bootstrap.cc:492] T 77eea4fef45b4b2fb4976e06fb15f386 P c5a13411dadf4739ba09a5da11f16926: Bootstrap starting.
I20260806 18:37:11.275483 4313 log.cc:824] T 77eea4fef45b4b2fb4976e06fb15f386 P 4d2a68a0b77a43619f6206aee728280d: Log is configured to *not* fsync() on all Append() calls
I20260806 18:37:11.276351 4315 log.cc:824] T 77eea4fef45b4b2fb4976e06fb15f386 P 5fdb1c79f09d42078b9fadc4624e6d0d: Log is configured to *not* fsync() on all Append() calls
I20260806 18:37:11.277556 4313 tablet_bootstrap.cc:492] T 77eea4fef45b4b2fb4976e06fb15f386 P 4d2a68a0b77a43619f6206aee728280d: No bootstrap required, opened a new log
I20260806 18:37:11.277648 4314 tablet_bootstrap.cc:654] T 77eea4fef45b4b2fb4976e06fb15f386 P c5a13411dadf4739ba09a5da11f16926: Neither blocks nor log segments found. Creating new log.
I20260806 18:37:11.277833 4313 ts_tablet_manager.cc:1397] T 77eea4fef45b4b2fb4976e06fb15f386 P 4d2a68a0b77a43619f6206aee728280d: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260806 18:37:11.278439 4315 tablet_bootstrap.cc:492] T 77eea4fef45b4b2fb4976e06fb15f386 P 5fdb1c79f09d42078b9fadc4624e6d0d: No bootstrap required, opened a new log
I20260806 18:37:11.278496 4314 log.cc:824] T 77eea4fef45b4b2fb4976e06fb15f386 P c5a13411dadf4739ba09a5da11f16926: Log is configured to *not* fsync() on all Append() calls
I20260806 18:37:11.278620 4315 ts_tablet_manager.cc:1397] T 77eea4fef45b4b2fb4976e06fb15f386 P 5fdb1c79f09d42078b9fadc4624e6d0d: Time spent bootstrapping tablet: real 0.005s user 0.003s sys 0.000s
I20260806 18:37:11.280462 4314 tablet_bootstrap.cc:492] T 77eea4fef45b4b2fb4976e06fb15f386 P c5a13411dadf4739ba09a5da11f16926: No bootstrap required, opened a new log
I20260806 18:37:11.280650 4314 ts_tablet_manager.cc:1397] T 77eea4fef45b4b2fb4976e06fb15f386 P c5a13411dadf4739ba09a5da11f16926: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260806 18:37:11.281075 4313 raft_consensus.cc:348] T 77eea4fef45b4b2fb4976e06fb15f386 P 4d2a68a0b77a43619f6206aee728280d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c5a13411dadf4739ba09a5da11f16926" member_type: VOTER last_known_addr { host: "127.19.194.3" port: 35207 } } peers { permanent_uuid: "5fdb1c79f09d42078b9fadc4624e6d0d" member_type: VOTER last_known_addr { host: "127.19.194.1" port: 36577 } } peers { permanent_uuid: "4d2a68a0b77a43619f6206aee728280d" member_type: VOTER last_known_addr { host: "127.19.194.2" port: 46303 } }
I20260806 18:37:11.281468 4313 raft_consensus.cc:374] T 77eea4fef45b4b2fb4976e06fb15f386 P 4d2a68a0b77a43619f6206aee728280d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 18:37:11.281558 4313 raft_consensus.cc:729] T 77eea4fef45b4b2fb4976e06fb15f386 P 4d2a68a0b77a43619f6206aee728280d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4d2a68a0b77a43619f6206aee728280d, State: Initialized, Role: FOLLOWER
I20260806 18:37:11.282007 4313 consensus_queue.cc:260] T 77eea4fef45b4b2fb4976e06fb15f386 P 4d2a68a0b77a43619f6206aee728280d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c5a13411dadf4739ba09a5da11f16926" member_type: VOTER last_known_addr { host: "127.19.194.3" port: 35207 } } peers { permanent_uuid: "5fdb1c79f09d42078b9fadc4624e6d0d" member_type: VOTER last_known_addr { host: "127.19.194.1" port: 36577 } } peers { permanent_uuid: "4d2a68a0b77a43619f6206aee728280d" member_type: VOTER last_known_addr { host: "127.19.194.2" port: 46303 } }
I20260806 18:37:11.282370 4315 raft_consensus.cc:348] T 77eea4fef45b4b2fb4976e06fb15f386 P 5fdb1c79f09d42078b9fadc4624e6d0d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c5a13411dadf4739ba09a5da11f16926" member_type: VOTER last_known_addr { host: "127.19.194.3" port: 35207 } } peers { permanent_uuid: "5fdb1c79f09d42078b9fadc4624e6d0d" member_type: VOTER last_known_addr { host: "127.19.194.1" port: 36577 } } peers { permanent_uuid: "4d2a68a0b77a43619f6206aee728280d" member_type: VOTER last_known_addr { host: "127.19.194.2" port: 46303 } }
I20260806 18:37:11.282617 4315 raft_consensus.cc:374] T 77eea4fef45b4b2fb4976e06fb15f386 P 5fdb1c79f09d42078b9fadc4624e6d0d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 18:37:11.282684 4315 raft_consensus.cc:729] T 77eea4fef45b4b2fb4976e06fb15f386 P 5fdb1c79f09d42078b9fadc4624e6d0d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5fdb1c79f09d42078b9fadc4624e6d0d, State: Initialized, Role: FOLLOWER
I20260806 18:37:11.283007 4313 ts_tablet_manager.cc:1428] T 77eea4fef45b4b2fb4976e06fb15f386 P 4d2a68a0b77a43619f6206aee728280d: Time spent starting tablet: real 0.005s user 0.006s sys 0.000s
I20260806 18:37:11.283149 4315 consensus_queue.cc:260] T 77eea4fef45b4b2fb4976e06fb15f386 P 5fdb1c79f09d42078b9fadc4624e6d0d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c5a13411dadf4739ba09a5da11f16926" member_type: VOTER last_known_addr { host: "127.19.194.3" port: 35207 } } peers { permanent_uuid: "5fdb1c79f09d42078b9fadc4624e6d0d" member_type: VOTER last_known_addr { host: "127.19.194.1" port: 36577 } } peers { permanent_uuid: "4d2a68a0b77a43619f6206aee728280d" member_type: VOTER last_known_addr { host: "127.19.194.2" port: 46303 } }
I20260806 18:37:11.283386 4149 heartbeater.cc:503] Master 127.19.194.62:36421 was elected leader, sending a full tablet report...
I20260806 18:37:11.284153 4315 ts_tablet_manager.cc:1428] T 77eea4fef45b4b2fb4976e06fb15f386 P 5fdb1c79f09d42078b9fadc4624e6d0d: Time spent starting tablet: real 0.005s user 0.006s sys 0.000s
I20260806 18:37:11.284639 4008 heartbeater.cc:503] Master 127.19.194.62:36421 was elected leader, sending a full tablet report...
I20260806 18:37:11.284628 4314 raft_consensus.cc:348] T 77eea4fef45b4b2fb4976e06fb15f386 P c5a13411dadf4739ba09a5da11f16926 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c5a13411dadf4739ba09a5da11f16926" member_type: VOTER last_known_addr { host: "127.19.194.3" port: 35207 } } peers { permanent_uuid: "5fdb1c79f09d42078b9fadc4624e6d0d" member_type: VOTER last_known_addr { host: "127.19.194.1" port: 36577 } } peers { permanent_uuid: "4d2a68a0b77a43619f6206aee728280d" member_type: VOTER last_known_addr { host: "127.19.194.2" port: 46303 } }
I20260806 18:37:11.284885 4314 raft_consensus.cc:374] T 77eea4fef45b4b2fb4976e06fb15f386 P c5a13411dadf4739ba09a5da11f16926 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 18:37:11.284924 4314 raft_consensus.cc:729] T 77eea4fef45b4b2fb4976e06fb15f386 P c5a13411dadf4739ba09a5da11f16926 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c5a13411dadf4739ba09a5da11f16926, State: Initialized, Role: FOLLOWER
I20260806 18:37:11.285266 4314 consensus_queue.cc:260] T 77eea4fef45b4b2fb4976e06fb15f386 P c5a13411dadf4739ba09a5da11f16926 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c5a13411dadf4739ba09a5da11f16926" member_type: VOTER last_known_addr { host: "127.19.194.3" port: 35207 } } peers { permanent_uuid: "5fdb1c79f09d42078b9fadc4624e6d0d" member_type: VOTER last_known_addr { host: "127.19.194.1" port: 36577 } } peers { permanent_uuid: "4d2a68a0b77a43619f6206aee728280d" member_type: VOTER last_known_addr { host: "127.19.194.2" port: 46303 } }
I20260806 18:37:11.286141 4314 ts_tablet_manager.cc:1428] T 77eea4fef45b4b2fb4976e06fb15f386 P c5a13411dadf4739ba09a5da11f16926: Time spent starting tablet: real 0.005s user 0.006s sys 0.000s
I20260806 18:37:11.286509 4293 heartbeater.cc:503] Master 127.19.194.62:36421 was elected leader, sending a full tablet report...
I20260806 18:37:11.344267 4319 raft_consensus.cc:482] T 77eea4fef45b4b2fb4976e06fb15f386 P 4d2a68a0b77a43619f6206aee728280d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 18:37:11.344458 4319 raft_consensus.cc:504] T 77eea4fef45b4b2fb4976e06fb15f386 P 4d2a68a0b77a43619f6206aee728280d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c5a13411dadf4739ba09a5da11f16926" member_type: VOTER last_known_addr { host: "127.19.194.3" port: 35207 } } peers { permanent_uuid: "5fdb1c79f09d42078b9fadc4624e6d0d" member_type: VOTER last_known_addr { host: "127.19.194.1" port: 36577 } } peers { permanent_uuid: "4d2a68a0b77a43619f6206aee728280d" member_type: VOTER last_known_addr { host: "127.19.194.2" port: 46303 } }
I20260806 18:37:11.345593 4319 leader_election.cc:288] T 77eea4fef45b4b2fb4976e06fb15f386 P 4d2a68a0b77a43619f6206aee728280d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c5a13411dadf4739ba09a5da11f16926 (127.19.194.3:35207), 5fdb1c79f09d42078b9fadc4624e6d0d (127.19.194.1:36577)
I20260806 18:37:11.347306 4320 raft_consensus.cc:482] T 77eea4fef45b4b2fb4976e06fb15f386 P 5fdb1c79f09d42078b9fadc4624e6d0d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 18:37:11.347491 4320 raft_consensus.cc:504] T 77eea4fef45b4b2fb4976e06fb15f386 P 5fdb1c79f09d42078b9fadc4624e6d0d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c5a13411dadf4739ba09a5da11f16926" member_type: VOTER last_known_addr { host: "127.19.194.3" port: 35207 } } peers { permanent_uuid: "5fdb1c79f09d42078b9fadc4624e6d0d" member_type: VOTER last_known_addr { host: "127.19.194.1" port: 36577 } } peers { permanent_uuid: "4d2a68a0b77a43619f6206aee728280d" member_type: VOTER last_known_addr { host: "127.19.194.2" port: 46303 } }
I20260806 18:37:11.348635 4320 leader_election.cc:288] T 77eea4fef45b4b2fb4976e06fb15f386 P 5fdb1c79f09d42078b9fadc4624e6d0d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c5a13411dadf4739ba09a5da11f16926 (127.19.194.3:35207), 4d2a68a0b77a43619f6206aee728280d (127.19.194.2:46303)
I20260806 18:37:11.350297 4151 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:37:11.345830 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.194.3:35207 (local address 127.19.194.2:46659)
0806 18:37:11.345952 (+ 122us) negotiation.cc:107] Waiting for socket to connect
0806 18:37:11.345969 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0806 18:37:11.346084 (+ 115us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:37:11.346343 (+ 259us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:37:11.346345 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:37:11.346363 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 18:37:11.346514 (+ 151us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:37:11.346520 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:37:11.347659 (+ 1139us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:37:11.347664 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:37:11.348805 (+ 1141us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:37:11.348816 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:37:11.349744 (+ 928us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:37:11.349747 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:37:11.349875 (+ 128us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:37:11.349910 (+ 35us) client_negotiation.cc:769] Sending connection context
0806 18:37:11.350032 (+ 122us) client_negotiation.cc:240] Negotiation successful
0806 18:37:11.350153 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":37}
I20260806 18:37:11.350570 4311 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:37:11.345937 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.194.2:40961 (local address 127.19.194.1:36577)
0806 18:37:11.346064 (+ 127us) server_negotiation.cc:206] Beginning negotiation
0806 18:37:11.346069 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 18:37:11.346430 (+ 361us) server_negotiation.cc:407] Connection header received
0806 18:37:11.346553 (+ 123us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:37:11.346559 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:37:11.346636 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:37:11.346703 (+ 67us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 18:37:11.347018 (+ 315us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:37:11.347779 (+ 761us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:37:11.349253 (+ 1474us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:37:11.349868 (+ 615us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:37:11.349967 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:37:11.350043 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0806 18:37:11.350370 (+ 327us) server_negotiation.cc:299] Negotiation successful
0806 18:37:11.350456 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260806 18:37:11.350953 4308 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:37:11.345942 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.194.2:46659 (local address 127.19.194.3:35207)
0806 18:37:11.346076 (+ 134us) server_negotiation.cc:206] Beginning negotiation
0806 18:37:11.346082 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 18:37:11.346099 (+ 17us) server_negotiation.cc:407] Connection header received
0806 18:37:11.346185 (+ 86us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:37:11.346188 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:37:11.346239 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:37:11.346325 (+ 86us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 18:37:11.346645 (+ 320us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:37:11.347511 (+ 866us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:37:11.348930 (+ 1419us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:37:11.349507 (+ 577us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:37:11.349616 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:37:11.349669 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0806 18:37:11.350734 (+ 1065us) server_negotiation.cc:299] Negotiation successful
0806 18:37:11.350826 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260806 18:37:11.351109 4322 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:37:11.345840 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.194.1:36577 (local address 127.19.194.2:40961)
0806 18:37:11.346330 (+ 490us) negotiation.cc:107] Waiting for socket to connect
0806 18:37:11.346345 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0806 18:37:11.346438 (+ 93us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:37:11.346740 (+ 302us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:37:11.346743 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:37:11.346760 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 18:37:11.346873 (+ 113us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:37:11.346880 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:37:11.347941 (+ 1061us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:37:11.347945 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:37:11.349118 (+ 1173us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:37:11.349129 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:37:11.350060 (+ 931us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:37:11.350066 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:37:11.350212 (+ 146us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:37:11.350236 (+ 24us) client_negotiation.cc:769] Sending connection context
0806 18:37:11.350878 (+ 642us) client_negotiation.cc:240] Negotiation successful
0806 18:37:11.350973 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":400,"mutex_wait_us":98,"thread_start_us":133,"threads_started":1}
I20260806 18:37:11.351702 4241 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "77eea4fef45b4b2fb4976e06fb15f386" candidate_uuid: "4d2a68a0b77a43619f6206aee728280d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c5a13411dadf4739ba09a5da11f16926" is_pre_election: true
I20260806 18:37:11.351908 3961 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "77eea4fef45b4b2fb4976e06fb15f386" candidate_uuid: "4d2a68a0b77a43619f6206aee728280d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5fdb1c79f09d42078b9fadc4624e6d0d" is_pre_election: true
I20260806 18:37:11.352000 4241 raft_consensus.cc:2454] T 77eea4fef45b4b2fb4976e06fb15f386 P c5a13411dadf4739ba09a5da11f16926 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4d2a68a0b77a43619f6206aee728280d in term 0.
I20260806 18:37:11.352080 3961 raft_consensus.cc:2454] T 77eea4fef45b4b2fb4976e06fb15f386 P 5fdb1c79f09d42078b9fadc4624e6d0d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4d2a68a0b77a43619f6206aee728280d in term 0.
I20260806 18:37:11.352509 4030 leader_election.cc:302] T 77eea4fef45b4b2fb4976e06fb15f386 P 4d2a68a0b77a43619f6206aee728280d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 4d2a68a0b77a43619f6206aee728280d, 5fdb1c79f09d42078b9fadc4624e6d0d, c5a13411dadf4739ba09a5da11f16926; no voters:
I20260806 18:37:11.352763 4319 raft_consensus.cc:2781] T 77eea4fef45b4b2fb4976e06fb15f386 P 4d2a68a0b77a43619f6206aee728280d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 18:37:11.352913 4319 raft_consensus.cc:482] T 77eea4fef45b4b2fb4976e06fb15f386 P 4d2a68a0b77a43619f6206aee728280d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 18:37:11.352999 4319 raft_consensus.cc:3037] T 77eea4fef45b4b2fb4976e06fb15f386 P 4d2a68a0b77a43619f6206aee728280d [term 0 FOLLOWER]: Advancing to term 1
I20260806 18:37:11.354228 4319 raft_consensus.cc:504] T 77eea4fef45b4b2fb4976e06fb15f386 P 4d2a68a0b77a43619f6206aee728280d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c5a13411dadf4739ba09a5da11f16926" member_type: VOTER last_known_addr { host: "127.19.194.3" port: 35207 } } peers { permanent_uuid: "5fdb1c79f09d42078b9fadc4624e6d0d" member_type: VOTER last_known_addr { host: "127.19.194.1" port: 36577 } } peers { permanent_uuid: "4d2a68a0b77a43619f6206aee728280d" member_type: VOTER last_known_addr { host: "127.19.194.2" port: 46303 } }
I20260806 18:37:11.354502 4312 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:37:11.349242 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.194.1:49837 (local address 127.19.194.2:46303)
0806 18:37:11.349499 (+ 257us) server_negotiation.cc:206] Beginning negotiation
0806 18:37:11.349503 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 18:37:11.349769 (+ 266us) server_negotiation.cc:407] Connection header received
0806 18:37:11.349900 (+ 131us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:37:11.349905 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:37:11.349958 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:37:11.350093 (+ 135us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 18:37:11.350466 (+ 373us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:37:11.351159 (+ 693us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:37:11.353255 (+ 2096us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:37:11.353837 (+ 582us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:37:11.353950 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:37:11.353997 (+ 47us) server_negotiation.cc:1035] Waiting for connection context
0806 18:37:11.354339 (+ 342us) server_negotiation.cc:299] Negotiation successful
0806 18:37:11.354402 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":188}
I20260806 18:37:11.354516 4323 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:37:11.348813 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.194.3:35207 (local address 127.19.194.1:59687)
0806 18:37:11.349059 (+ 246us) negotiation.cc:107] Waiting for socket to connect
0806 18:37:11.349073 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0806 18:37:11.349329 (+ 256us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:37:11.350576 (+ 1247us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:37:11.350581 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:37:11.350603 (+ 22us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 18:37:11.350717 (+ 114us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:37:11.350724 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:37:11.351862 (+ 1138us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:37:11.351865 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:37:11.352941 (+ 1076us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:37:11.352951 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:37:11.354016 (+ 1065us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:37:11.354029 (+ 13us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:37:11.354159 (+ 130us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:37:11.354192 (+ 33us) client_negotiation.cc:769] Sending connection context
0806 18:37:11.354285 (+ 93us) client_negotiation.cc:240] Negotiation successful
0806 18:37:11.354362 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":164,"mutex_wait_us":21,"thread_start_us":90,"threads_started":1}
I20260806 18:37:11.354532 4324 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:37:11.349192 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.194.2:46303 (local address 127.19.194.1:49837)
0806 18:37:11.349686 (+ 494us) negotiation.cc:107] Waiting for socket to connect
0806 18:37:11.349701 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0806 18:37:11.349816 (+ 115us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:37:11.350193 (+ 377us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:37:11.350196 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:37:11.350213 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 18:37:11.350347 (+ 134us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:37:11.350357 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:37:11.351512 (+ 1155us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:37:11.351515 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:37:11.353120 (+ 1605us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:37:11.353128 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:37:11.354094 (+ 966us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:37:11.354097 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:37:11.354197 (+ 100us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:37:11.354213 (+ 16us) client_negotiation.cc:769] Sending connection context
0806 18:37:11.354307 (+ 94us) client_negotiation.cc:240] Negotiation successful
0806 18:37:11.354379 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":386,"mutex_wait_us":30,"thread_start_us":86,"threads_started":1}
I20260806 18:37:11.354961 4325 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:37:11.349501 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.194.1:59687 (local address 127.19.194.3:35207)
0806 18:37:11.350295 (+ 794us) server_negotiation.cc:206] Beginning negotiation
0806 18:37:11.350304 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0806 18:37:11.350321 (+ 17us) server_negotiation.cc:407] Connection header received
0806 18:37:11.350365 (+ 44us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:37:11.350369 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:37:11.350415 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:37:11.350820 (+ 405us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 18:37:11.350902 (+ 82us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:37:11.351723 (+ 821us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:37:11.353255 (+ 1532us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:37:11.353887 (+ 632us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:37:11.353994 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:37:11.354049 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0806 18:37:11.354708 (+ 659us) server_negotiation.cc:299] Negotiation successful
0806 18:37:11.354794 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":721,"thread_start_us":94,"threads_started":1}
I20260806 18:37:11.355201 4099 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "77eea4fef45b4b2fb4976e06fb15f386" candidate_uuid: "5fdb1c79f09d42078b9fadc4624e6d0d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4d2a68a0b77a43619f6206aee728280d" is_pre_election: true
I20260806 18:37:11.355428 4099 raft_consensus.cc:2379] T 77eea4fef45b4b2fb4976e06fb15f386 P 4d2a68a0b77a43619f6206aee728280d [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5fdb1c79f09d42078b9fadc4624e6d0d in current term 1: Already voted for candidate 4d2a68a0b77a43619f6206aee728280d in this term.
I20260806 18:37:11.355396 4241 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "77eea4fef45b4b2fb4976e06fb15f386" candidate_uuid: "5fdb1c79f09d42078b9fadc4624e6d0d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c5a13411dadf4739ba09a5da11f16926" is_pre_election: true
I20260806 18:37:11.355520 4241 raft_consensus.cc:2454] T 77eea4fef45b4b2fb4976e06fb15f386 P c5a13411dadf4739ba09a5da11f16926 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5fdb1c79f09d42078b9fadc4624e6d0d in term 0.
I20260806 18:37:11.355536 4240 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "77eea4fef45b4b2fb4976e06fb15f386" candidate_uuid: "4d2a68a0b77a43619f6206aee728280d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c5a13411dadf4739ba09a5da11f16926"
I20260806 18:37:11.355597 3961 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "77eea4fef45b4b2fb4976e06fb15f386" candidate_uuid: "4d2a68a0b77a43619f6206aee728280d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5fdb1c79f09d42078b9fadc4624e6d0d"
I20260806 18:37:11.355619 4240 raft_consensus.cc:3037] T 77eea4fef45b4b2fb4976e06fb15f386 P c5a13411dadf4739ba09a5da11f16926 [term 0 FOLLOWER]: Advancing to term 1
I20260806 18:37:11.355705 3961 raft_consensus.cc:3037] T 77eea4fef45b4b2fb4976e06fb15f386 P 5fdb1c79f09d42078b9fadc4624e6d0d [term 0 FOLLOWER]: Advancing to term 1
I20260806 18:37:11.355937 3892 leader_election.cc:302] T 77eea4fef45b4b2fb4976e06fb15f386 P 5fdb1c79f09d42078b9fadc4624e6d0d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5fdb1c79f09d42078b9fadc4624e6d0d, c5a13411dadf4739ba09a5da11f16926; no voters:
I20260806 18:37:11.356822 3961 raft_consensus.cc:2454] T 77eea4fef45b4b2fb4976e06fb15f386 P 5fdb1c79f09d42078b9fadc4624e6d0d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4d2a68a0b77a43619f6206aee728280d in term 1.
I20260806 18:37:11.356920 4240 raft_consensus.cc:2454] T 77eea4fef45b4b2fb4976e06fb15f386 P c5a13411dadf4739ba09a5da11f16926 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4d2a68a0b77a43619f6206aee728280d in term 1.
I20260806 18:37:11.356927 4320 raft_consensus.cc:2741] T 77eea4fef45b4b2fb4976e06fb15f386 P 5fdb1c79f09d42078b9fadc4624e6d0d [term 1 FOLLOWER]: Leader pre-election decision vote started in defunct term 0: won
I20260806 18:37:11.357038 4319 leader_election.cc:288] T 77eea4fef45b4b2fb4976e06fb15f386 P 4d2a68a0b77a43619f6206aee728280d [CANDIDATE]: Term 1 election: Requested vote from peers c5a13411dadf4739ba09a5da11f16926 (127.19.194.3:35207), 5fdb1c79f09d42078b9fadc4624e6d0d (127.19.194.1:36577)
I20260806 18:37:11.357218 4030 leader_election.cc:302] T 77eea4fef45b4b2fb4976e06fb15f386 P 4d2a68a0b77a43619f6206aee728280d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4d2a68a0b77a43619f6206aee728280d, 5fdb1c79f09d42078b9fadc4624e6d0d; no voters:
I20260806 18:37:11.357769 4319 raft_consensus.cc:2781] T 77eea4fef45b4b2fb4976e06fb15f386 P 4d2a68a0b77a43619f6206aee728280d [term 1 FOLLOWER]: Leader election won for term 1
I20260806 18:37:11.357954 4319 raft_consensus.cc:686] T 77eea4fef45b4b2fb4976e06fb15f386 P 4d2a68a0b77a43619f6206aee728280d [term 1 LEADER]: Becoming Leader. State: Replica: 4d2a68a0b77a43619f6206aee728280d, State: Running, Role: LEADER
I20260806 18:37:11.358251 4319 consensus_queue.cc:237] T 77eea4fef45b4b2fb4976e06fb15f386 P 4d2a68a0b77a43619f6206aee728280d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c5a13411dadf4739ba09a5da11f16926" member_type: VOTER last_known_addr { host: "127.19.194.3" port: 35207 } } peers { permanent_uuid: "5fdb1c79f09d42078b9fadc4624e6d0d" member_type: VOTER last_known_addr { host: "127.19.194.1" port: 36577 } } peers { permanent_uuid: "4d2a68a0b77a43619f6206aee728280d" member_type: VOTER last_known_addr { host: "127.19.194.2" port: 46303 } }
I20260806 18:37:11.361217 3808 catalog_manager.cc:5495] T 77eea4fef45b4b2fb4976e06fb15f386 P 4d2a68a0b77a43619f6206aee728280d reported cstate change: term changed from 0 to 1, leader changed from <none> to 4d2a68a0b77a43619f6206aee728280d (127.19.194.2). New cstate: current_term: 1 leader_uuid: "4d2a68a0b77a43619f6206aee728280d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c5a13411dadf4739ba09a5da11f16926" member_type: VOTER last_known_addr { host: "127.19.194.3" port: 35207 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5fdb1c79f09d42078b9fadc4624e6d0d" member_type: VOTER last_known_addr { host: "127.19.194.1" port: 36577 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4d2a68a0b77a43619f6206aee728280d" member_type: VOTER last_known_addr { host: "127.19.194.2" port: 46303 } health_report { overall_health: HEALTHY } } }
I20260806 18:37:11.384914 4312 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:37:11.381452 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45158 (local address 127.19.194.2:46303)
0806 18:37:11.381607 (+ 155us) server_negotiation.cc:206] Beginning negotiation
0806 18:37:11.381611 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 18:37:11.381632 (+ 21us) server_negotiation.cc:407] Connection header received
0806 18:37:11.381786 (+ 154us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:37:11.381788 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:37:11.381840 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:37:11.381937 (+ 97us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 18:37:11.382156 (+ 219us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:37:11.382643 (+ 487us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:37:11.383530 (+ 887us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:37:11.383733 (+ 203us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:37:11.383825 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:37:11.384076 (+ 251us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 18:37:11.384465 (+ 389us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 18:37:11.384553 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0806 18:37:11.384693 (+ 140us) server_negotiation.cc:299] Negotiation successful
0806 18:37:11.384765 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260806 18:37:11.390456 4240 raft_consensus.cc:1261] T 77eea4fef45b4b2fb4976e06fb15f386 P c5a13411dadf4739ba09a5da11f16926 [term 1 FOLLOWER]: Refusing update from remote peer 4d2a68a0b77a43619f6206aee728280d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 18:37:11.390455 3961 raft_consensus.cc:1261] T 77eea4fef45b4b2fb4976e06fb15f386 P 5fdb1c79f09d42078b9fadc4624e6d0d [term 1 FOLLOWER]: Refusing update from remote peer 4d2a68a0b77a43619f6206aee728280d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 18:37:11.391077 4327 consensus_queue.cc:1035] T 77eea4fef45b4b2fb4976e06fb15f386 P 4d2a68a0b77a43619f6206aee728280d [LEADER]: Connected to new peer: Peer: permanent_uuid: "5fdb1c79f09d42078b9fadc4624e6d0d" member_type: VOTER last_known_addr { host: "127.19.194.1" port: 36577 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 18:37:11.391314 4319 consensus_queue.cc:1035] T 77eea4fef45b4b2fb4976e06fb15f386 P 4d2a68a0b77a43619f6206aee728280d [LEADER]: Connected to new peer: Peer: permanent_uuid: "c5a13411dadf4739ba09a5da11f16926" member_type: VOTER last_known_addr { host: "127.19.194.3" port: 35207 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 18:37:11.398684 4331 mvcc.cc:205] Tried to move back new op lower bound from 7315625702968451072 to 7315625702845943808. Current Snapshot: MvccSnapshot[applied={T|T < 7315625702968451072}]
I20260806 18:37:11.407532 3808 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:44482:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 18:37:11.407712 3808 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:44482:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 18:37:11.410295 3808 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 34dfff54533849c68cf3a4743721aeb5: Sending DeleteTablet for 3 replicas of tablet 77eea4fef45b4b2fb4976e06fb15f386
I20260806 18:37:11.411043 3940 tablet_service.cc:1514] Processing DeleteTablet for tablet 77eea4fef45b4b2fb4976e06fb15f386 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 18:37:11 UTC) from {username='kudu', principal='kudu/127.19.194.62@KRBTEST.COM'} at 127.0.0.1:46596
I20260806 18:37:11.411077 4219 tablet_service.cc:1514] Processing DeleteTablet for tablet 77eea4fef45b4b2fb4976e06fb15f386 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 18:37:11 UTC) from {username='kudu', principal='kudu/127.19.194.62@KRBTEST.COM'} at 127.0.0.1:51352
I20260806 18:37:11.411129 4079 tablet_service.cc:1514] Processing DeleteTablet for tablet 77eea4fef45b4b2fb4976e06fb15f386 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 18:37:11 UTC) from {username='kudu', principal='kudu/127.19.194.62@KRBTEST.COM'} at 127.0.0.1:45146
I20260806 18:37:11.411218 20232 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJW69tn/build/debug/bin/kudu with pid 3873
I20260806 18:37:11.412580 4341 tablet_replica.cc:330] T 77eea4fef45b4b2fb4976e06fb15f386 P 4d2a68a0b77a43619f6206aee728280d: stopping tablet replica
I20260806 18:37:11.412920 4341 raft_consensus.cc:2229] T 77eea4fef45b4b2fb4976e06fb15f386 P 4d2a68a0b77a43619f6206aee728280d [term 1 LEADER]: Raft consensus shutting down.
I20260806 18:37:11.413410 4341 raft_consensus.cc:2258] T 77eea4fef45b4b2fb4976e06fb15f386 P 4d2a68a0b77a43619f6206aee728280d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 18:37:11.413847 4340 tablet_replica.cc:330] T 77eea4fef45b4b2fb4976e06fb15f386 P c5a13411dadf4739ba09a5da11f16926: stopping tablet replica
I20260806 18:37:11.414204 4340 raft_consensus.cc:2229] T 77eea4fef45b4b2fb4976e06fb15f386 P c5a13411dadf4739ba09a5da11f16926 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 18:37:11.414577 4340 raft_consensus.cc:2258] T 77eea4fef45b4b2fb4976e06fb15f386 P c5a13411dadf4739ba09a5da11f16926 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 18:37:11.415642 4341 ts_tablet_manager.cc:1905] T 77eea4fef45b4b2fb4976e06fb15f386 P 4d2a68a0b77a43619f6206aee728280d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 18:37:11.416110 4340 ts_tablet_manager.cc:1905] T 77eea4fef45b4b2fb4976e06fb15f386 P c5a13411dadf4739ba09a5da11f16926: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 18:37:11.418956 4340 ts_tablet_manager.cc:1918] T 77eea4fef45b4b2fb4976e06fb15f386 P c5a13411dadf4739ba09a5da11f16926: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 18:37:11.419137 4340 log.cc:1196] T 77eea4fef45b4b2fb4976e06fb15f386 P c5a13411dadf4739ba09a5da11f16926: Deleting WAL directory at /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786041246094603-20232-0/minicluster-data/ts-2/wal/wals/77eea4fef45b4b2fb4976e06fb15f386
I20260806 18:37:11.419319 4341 ts_tablet_manager.cc:1918] T 77eea4fef45b4b2fb4976e06fb15f386 P 4d2a68a0b77a43619f6206aee728280d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 18:37:11.419507 4341 log.cc:1196] T 77eea4fef45b4b2fb4976e06fb15f386 P 4d2a68a0b77a43619f6206aee728280d: Deleting WAL directory at /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786041246094603-20232-0/minicluster-data/ts-1/wal/wals/77eea4fef45b4b2fb4976e06fb15f386
I20260806 18:37:11.419932 4340 ts_tablet_manager.cc:1939] T 77eea4fef45b4b2fb4976e06fb15f386 P c5a13411dadf4739ba09a5da11f16926: Deleting consensus metadata
I20260806 18:37:11.419938 4341 ts_tablet_manager.cc:1939] T 77eea4fef45b4b2fb4976e06fb15f386 P 4d2a68a0b77a43619f6206aee728280d: Deleting consensus metadata
W20260806 18:37:11.420979 3793 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.19.194.1:36577 (error 108)
I20260806 18:37:11.421262 3794 catalog_manager.cc:4841] TS c5a13411dadf4739ba09a5da11f16926 (127.19.194.3:35207): tablet 77eea4fef45b4b2fb4976e06fb15f386 (table test-table [id=cd76f96f476143f6a38dddbaef9c6ea8]) successfully deleted
I20260806 18:37:11.421490 3794 catalog_manager.cc:4841] TS 4d2a68a0b77a43619f6206aee728280d (127.19.194.2:46303): tablet 77eea4fef45b4b2fb4976e06fb15f386 (table test-table [id=cd76f96f476143f6a38dddbaef9c6ea8]) successfully deleted
I20260806 18:37:11.421875 20232 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJW69tn/build/debug/bin/kudu with pid 4013
I20260806 18:37:11.422233 4306 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:37:11.421920 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.194.1:36577 (local address 127.0.0.1:46600)
0806 18:37:11.422088 (+ 168us) negotiation.cc:107] Waiting for socket to connect
0806 18:37:11.422151 (+ 63us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.194.1:36577: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":93}
W20260806 18:37:11.422900 3793 catalog_manager.cc:4584] TS 5fdb1c79f09d42078b9fadc4624e6d0d (127.19.194.1:36577): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 77eea4fef45b4b2fb4976e06fb15f386: Network error: Client connection negotiation failed: client connection to 127.19.194.1:36577: connect: Connection refused (error 111)
I20260806 18:37:11.429517 20232 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJW69tn/build/debug/bin/kudu with pid 4154
I20260806 18:37:11.436739 20232 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJW69tn/build/debug/bin/kudu with pid 3776
I20260806 18:37:31.444296 20232 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJW69tn/build/debug/bin/kudu
/tmp/dist-test-taskJW69tn/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786041246094603-20232-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786041246094603-20232-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786041246094603-20232-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786041246094603-20232-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.19.194.62:36421
--webserver_interface=127.19.194.62
--webserver_port=39513
--keytab_file=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786041246094603-20232-0/krb5kdc/kudu_127.19.194.62.keytab
--principal=kudu/127.19.194.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.19.194.20:36777
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.19.194.62:36421
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--tsk_private_key_password_cmd=echo foo with env {KRB5CCNAME=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786041246094603-20232-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786041246094603-20232-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786041246094603-20232-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 18:37:31.553834 4343 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 18:37:31.554072 4343 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 18:37:31.554109 4343 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 18:37:31.558059 4343 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260806 18:37:31.558109 4343 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260806 18:37:31.558126 4343 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260806 18:37:31.558140 4343 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 18:37:31.558205 4343 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 18:37:31.558238 4343 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 18:37:31.558277 4343 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 18:37:31.558329 4343 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 18:37:31.563469 4343 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.194.20:36777
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786041246094603-20232-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786041246094603-20232-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_private_key_password_cmd=echo foo
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.19.194.62:36421
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786041246094603-20232-0/krb5kdc/kudu_127.19.194.62.keytab
--principal=kudu/127.19.194.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.19.194.62:36421
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786041246094603-20232-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.19.194.62
--webserver_port=39513
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4343
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786041246094603-20232-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 17:59:13 UTC on ccycloud.quasar-wvsnzx.root.comops.site
build id 4571028
I20260806 18:37:31.564484 4343 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 18:37:31.565505 4343 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260806 18:37:31.579278 4343 server_base.cc:1034] running on GCE node
W20260806 18:37:31.579247 4352 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:37:31.579241 4349 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:37:31.579480 4350 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 18:37:31.580134 4343 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 18:37:31.581240 4343 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 18:37:31.582404 4343 hybrid_clock.cc:648] HybridClock initialized: now 1786041451582382 us; error 34 us; skew 500 ppm
Aug 06 18:37:31 dist-test-slave-1xrh krb5kdc[3760](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041451, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.194.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 18:37:31.591444 4343 init.cc:377] Logged in from keytab as kudu/127.19.194.62@KRBTEST.COM (short username kudu)
I20260806 18:37:31.592360 4343 webserver.cc:460] Webserver started at http://127.19.194.62:39513/ using document root <none> and password file <none>
I20260806 18:37:31.592935 4343 fs_manager.cc:359] Metadata directory not provided
I20260806 18:37:31.593034 4343 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 18:37:31.596839 4343 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.003s sys 0.000s
I20260806 18:37:31.598875 4359 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 18:37:31.600077 4343 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260806 18:37:31.600219 4343 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786041246094603-20232-0/minicluster-data/master-0/data,/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786041246094603-20232-0/minicluster-data/master-0/wal
uuid: "34dfff54533849c68cf3a4743721aeb5"
format_stamp: "Formatted at 2026-08-06 18:37:10 on dist-test-slave-1xrh"
server_key: "6ac8ff971ade94db00b2ef63baf3c45d"
server_key_iv: "c0bad579135a7450618b2f3d295e550d"
server_key_version: "encryptionkey@0"
I20260806 18:37:31.600802 4343 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786041246094603-20232-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786041246094603-20232-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786041246094603-20232-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 18:37:31.610805 4343 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 18:37:31.613374 4343 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 18:37:31.613579 4343 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 18:37:31.620714 4343 rpc_server.cc:307] RPC server started. Bound to: 127.19.194.62:36421
I20260806 18:37:31.621016 4419 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.194.62:36421 every 8 connection(s)
I20260806 18:37:31.621848 4343 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786041246094603-20232-0/minicluster-data/master-0/data/info.pb
I20260806 18:37:31.625937 4420 sys_catalog.cc:263] Verifying existing consensus state
I20260806 18:37:31.627562 4420 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 34dfff54533849c68cf3a4743721aeb5: Bootstrap starting.
I20260806 18:37:31.631349 20232 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJW69tn/build/debug/bin/kudu as pid 4343
I20260806 18:37:31.631772 20232 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJW69tn/build/debug/bin/kudu
/tmp/dist-test-taskJW69tn/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786041246094603-20232-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786041246094603-20232-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786041246094603-20232-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786041246094603-20232-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.19.194.1:36577
--local_ip_for_outbound_sockets=127.19.194.1
--tserver_master_addrs=127.19.194.62:36421
--webserver_port=40163
--webserver_interface=127.19.194.1
--keytab_file=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786041246094603-20232-0/krb5kdc/kudu_127.19.194.1.keytab
--principal=kudu/127.19.194.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.19.194.20:36777
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786041246094603-20232-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786041246094603-20232-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786041246094603-20232-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260806 18:37:31.646188 4420 log.cc:824] T 00000000000000000000000000000000 P 34dfff54533849c68cf3a4743721aeb5: Log is configured to *not* fsync() on all Append() calls
I20260806 18:37:31.654963 4420 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 34dfff54533849c68cf3a4743721aeb5: Bootstrap replayed 1/1 log segments. Stats: ops{read=8 overwritten=0 applied=8 ignored=0} inserts{seen=5 ignored=0} mutations{seen=4 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260806 18:37:31.655395 4420 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 34dfff54533849c68cf3a4743721aeb5: Bootstrap complete.
I20260806 18:37:31.659507 4420 raft_consensus.cc:348] T 00000000000000000000000000000000 P 34dfff54533849c68cf3a4743721aeb5 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "34dfff54533849c68cf3a4743721aeb5" member_type: VOTER last_known_addr { host: "127.19.194.62" port: 36421 } }
I20260806 18:37:31.659922 4420 raft_consensus.cc:729] T 00000000000000000000000000000000 P 34dfff54533849c68cf3a4743721aeb5 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 34dfff54533849c68cf3a4743721aeb5, State: Initialized, Role: FOLLOWER
I20260806 18:37:31.660452 4420 consensus_queue.cc:260] T 00000000000000000000000000000000 P 34dfff54533849c68cf3a4743721aeb5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "34dfff54533849c68cf3a4743721aeb5" member_type: VOTER last_known_addr { host: "127.19.194.62" port: 36421 } }
I20260806 18:37:31.660625 4420 raft_consensus.cc:388] T 00000000000000000000000000000000 P 34dfff54533849c68cf3a4743721aeb5 [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 18:37:31.660724 4420 raft_consensus.cc:482] T 00000000000000000000000000000000 P 34dfff54533849c68cf3a4743721aeb5 [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 18:37:31.660842 4420 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 34dfff54533849c68cf3a4743721aeb5 [term 1 FOLLOWER]: Advancing to term 2
I20260806 18:37:31.662678 4420 raft_consensus.cc:504] T 00000000000000000000000000000000 P 34dfff54533849c68cf3a4743721aeb5 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "34dfff54533849c68cf3a4743721aeb5" member_type: VOTER last_known_addr { host: "127.19.194.62" port: 36421 } }
I20260806 18:37:31.663009 4420 leader_election.cc:302] T 00000000000000000000000000000000 P 34dfff54533849c68cf3a4743721aeb5 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 34dfff54533849c68cf3a4743721aeb5; no voters:
I20260806 18:37:31.663357 4420 leader_election.cc:288] T 00000000000000000000000000000000 P 34dfff54533849c68cf3a4743721aeb5 [CANDIDATE]: Term 2 election: Requested vote from peers
I20260806 18:37:31.663465 4424 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 34dfff54533849c68cf3a4743721aeb5 [term 2 FOLLOWER]: Leader election won for term 2
I20260806 18:37:31.663673 4424 raft_consensus.cc:686] T 00000000000000000000000000000000 P 34dfff54533849c68cf3a4743721aeb5 [term 2 LEADER]: Becoming Leader. State: Replica: 34dfff54533849c68cf3a4743721aeb5, State: Running, Role: LEADER
I20260806 18:37:31.664044 4424 consensus_queue.cc:237] T 00000000000000000000000000000000 P 34dfff54533849c68cf3a4743721aeb5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "34dfff54533849c68cf3a4743721aeb5" member_type: VOTER last_known_addr { host: "127.19.194.62" port: 36421 } }
I20260806 18:37:31.664599 4420 sys_catalog.cc:564] T 00000000000000000000000000000000 P 34dfff54533849c68cf3a4743721aeb5 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 18:37:31.665405 4425 sys_catalog.cc:455] T 00000000000000000000000000000000 P 34dfff54533849c68cf3a4743721aeb5 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "34dfff54533849c68cf3a4743721aeb5" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "34dfff54533849c68cf3a4743721aeb5" member_type: VOTER last_known_addr { host: "127.19.194.62" port: 36421 } } }
I20260806 18:37:31.665546 4425 sys_catalog.cc:458] T 00000000000000000000000000000000 P 34dfff54533849c68cf3a4743721aeb5 [sys.catalog]: This master's current role is: LEADER
I20260806 18:37:31.665556 4426 sys_catalog.cc:455] T 00000000000000000000000000000000 P 34dfff54533849c68cf3a4743721aeb5 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 34dfff54533849c68cf3a4743721aeb5. Latest consensus state: current_term: 2 leader_uuid: "34dfff54533849c68cf3a4743721aeb5" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "34dfff54533849c68cf3a4743721aeb5" member_type: VOTER last_known_addr { host: "127.19.194.62" port: 36421 } } }
I20260806 18:37:31.665920 4426 sys_catalog.cc:458] T 00000000000000000000000000000000 P 34dfff54533849c68cf3a4743721aeb5 [sys.catalog]: This master's current role is: LEADER
I20260806 18:37:31.666420 4429 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 18:37:31.671149 4429 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 18:37:31.671949 4429 catalog_manager.cc:1259] Loaded cluster ID: 9dc90a31f80745688367d0fd1a89a281
I20260806 18:37:31.672016 4429 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 18:37:31.674537 4429 catalog_manager.cc:1504] Loading token signing keys...
I20260806 18:37:31.675246 4429 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 34dfff54533849c68cf3a4743721aeb5: Loaded TSK: 0
I20260806 18:37:31.690900 4429 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 34dfff54533849c68cf3a4743721aeb5: Generated new TSK 1
I20260806 18:37:31.691207 4429 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260806 18:37:31.750172 4422 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 18:37:31.750403 4422 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 18:37:31.750443 4422 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 18:37:31.754472 4422 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 18:37:31.754524 4422 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 18:37:31.754578 4422 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.19.194.1
I20260806 18:37:31.759784 4422 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.194.20:36777
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786041246094603-20232-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786041246094603-20232-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786041246094603-20232-0/krb5kdc/kudu_127.19.194.1.keytab
--principal=kudu/127.19.194.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.19.194.1:36577
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786041246094603-20232-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.19.194.1
--webserver_port=40163
--tserver_master_addrs=127.19.194.62:36421
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4422
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.19.194.1
--log_dir=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786041246094603-20232-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 17:59:13 UTC on ccycloud.quasar-wvsnzx.root.comops.site
build id 4571028
I20260806 18:37:31.760716 4422 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 18:37:31.761633 4422 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 18:37:31.771237 4451 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:37:31.771238 4450 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 18:37:31.771456 4422 server_base.cc:1034] running on GCE node
W20260806 18:37:31.771682 4453 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 18:37:31.772061 4422 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 18:37:31.772723 4422 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 18:37:31.773905 4422 hybrid_clock.cc:648] HybridClock initialized: now 1786041451773881 us; error 50 us; skew 500 ppm
Aug 06 18:37:31 dist-test-slave-1xrh krb5kdc[3760](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041451, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.194.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 18:37:31.776684 4422 init.cc:377] Logged in from keytab as kudu/127.19.194.1@KRBTEST.COM (short username kudu)
I20260806 18:37:31.777459 4422 webserver.cc:460] Webserver started at http://127.19.194.1:40163/ using document root <none> and password file <none>
I20260806 18:37:31.777977 4422 fs_manager.cc:359] Metadata directory not provided
I20260806 18:37:31.778052 4422 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 18:37:31.781852 4422 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.002s sys 0.000s
I20260806 18:37:31.784145 4461 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 18:37:31.785375 4422 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260806 18:37:31.785523 4422 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786041246094603-20232-0/minicluster-data/ts-0/data,/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786041246094603-20232-0/minicluster-data/ts-0/wal
uuid: "5fdb1c79f09d42078b9fadc4624e6d0d"
format_stamp: "Formatted at 2026-08-06 18:37:10 on dist-test-slave-1xrh"
server_key: "18754be74733259eab711d3b24a590eb"
server_key_iv: "532b38fc980d1832027c7f020488d752"
server_key_version: "encryptionkey@0"
I20260806 18:37:31.786058 4422 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786041246094603-20232-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786041246094603-20232-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786041246094603-20232-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 18:37:31.802603 4422 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 18:37:31.806833 4422 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 18:37:31.807096 4422 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 18:37:31.807901 4422 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 18:37:31.809567 4468 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260806 18:37:31.814384 4422 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260806 18:37:31.814448 4422 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.005s user 0.001s sys 0.000s
I20260806 18:37:31.814513 4422 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260806 18:37:31.816962 4422 ts_tablet_manager.cc:610] Registered 1 tablets
I20260806 18:37:31.817019 4422 ts_tablet_manager.cc:589] Time spent register tablets: real 0.003s user 0.001s sys 0.000s
I20260806 18:37:31.817273 4468 tablet_bootstrap.cc:492] T 77eea4fef45b4b2fb4976e06fb15f386 P 5fdb1c79f09d42078b9fadc4624e6d0d: Bootstrap starting.
I20260806 18:37:31.828094 4422 rpc_server.cc:307] RPC server started. Bound to: 127.19.194.1:36577
I20260806 18:37:31.828155 4582 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.194.1:36577 every 8 connection(s)
I20260806 18:37:31.829250 4422 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786041246094603-20232-0/minicluster-data/ts-0/data/info.pb
I20260806 18:37:31.839370 20232 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJW69tn/build/debug/bin/kudu as pid 4422
I20260806 18:37:31.839761 4468 log.cc:824] T 77eea4fef45b4b2fb4976e06fb15f386 P 5fdb1c79f09d42078b9fadc4624e6d0d: Log is configured to *not* fsync() on all Append() calls
I20260806 18:37:31.839956 20232 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJW69tn/build/debug/bin/kudu
/tmp/dist-test-taskJW69tn/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786041246094603-20232-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786041246094603-20232-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786041246094603-20232-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786041246094603-20232-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.19.194.2:46303
--local_ip_for_outbound_sockets=127.19.194.2
--tserver_master_addrs=127.19.194.62:36421
--webserver_port=38271
--webserver_interface=127.19.194.2
--keytab_file=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786041246094603-20232-0/krb5kdc/kudu_127.19.194.2.keytab
--principal=kudu/127.19.194.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.19.194.20:36777
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786041246094603-20232-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786041246094603-20232-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786041246094603-20232-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 06 18:37:31 dist-test-slave-1xrh krb5kdc[3760](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041451, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.194.1@KRBTEST.COM for kudu/127.19.194.62@KRBTEST.COM
I20260806 18:37:31.849226 4468 tablet_bootstrap.cc:492] T 77eea4fef45b4b2fb4976e06fb15f386 P 5fdb1c79f09d42078b9fadc4624e6d0d: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260806 18:37:31.849715 4468 tablet_bootstrap.cc:492] T 77eea4fef45b4b2fb4976e06fb15f386 P 5fdb1c79f09d42078b9fadc4624e6d0d: Bootstrap complete.
I20260806 18:37:31.851481 4468 ts_tablet_manager.cc:1397] T 77eea4fef45b4b2fb4976e06fb15f386 P 5fdb1c79f09d42078b9fadc4624e6d0d: Time spent bootstrapping tablet: real 0.034s user 0.029s sys 0.004s
I20260806 18:37:31.855276 4468 raft_consensus.cc:348] T 77eea4fef45b4b2fb4976e06fb15f386 P 5fdb1c79f09d42078b9fadc4624e6d0d [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c5a13411dadf4739ba09a5da11f16926" member_type: VOTER last_known_addr { host: "127.19.194.3" port: 35207 } } peers { permanent_uuid: "5fdb1c79f09d42078b9fadc4624e6d0d" member_type: VOTER last_known_addr { host: "127.19.194.1" port: 36577 } } peers { permanent_uuid: "4d2a68a0b77a43619f6206aee728280d" member_type: VOTER last_known_addr { host: "127.19.194.2" port: 46303 } }
I20260806 18:37:31.855983 4468 raft_consensus.cc:729] T 77eea4fef45b4b2fb4976e06fb15f386 P 5fdb1c79f09d42078b9fadc4624e6d0d [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5fdb1c79f09d42078b9fadc4624e6d0d, State: Initialized, Role: FOLLOWER
I20260806 18:37:31.856593 4468 consensus_queue.cc:260] T 77eea4fef45b4b2fb4976e06fb15f386 P 5fdb1c79f09d42078b9fadc4624e6d0d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c5a13411dadf4739ba09a5da11f16926" member_type: VOTER last_known_addr { host: "127.19.194.3" port: 35207 } } peers { permanent_uuid: "5fdb1c79f09d42078b9fadc4624e6d0d" member_type: VOTER last_known_addr { host: "127.19.194.1" port: 36577 } } peers { permanent_uuid: "4d2a68a0b77a43619f6206aee728280d" member_type: VOTER last_known_addr { host: "127.19.194.2" port: 46303 } }
I20260806 18:37:31.857023 4586 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:37:31.833145 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.194.1:60547 (local address 127.19.194.62:36421)
0806 18:37:31.833528 (+ 383us) server_negotiation.cc:206] Beginning negotiation
0806 18:37:31.833550 (+ 22us) server_negotiation.cc:399] Waiting for connection header
0806 18:37:31.833597 (+ 47us) server_negotiation.cc:407] Connection header received
0806 18:37:31.834153 (+ 556us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:37:31.834174 (+ 21us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:37:31.834620 (+ 446us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:37:31.834981 (+ 361us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:37:31.836499 (+ 1518us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:37:31.838156 (+ 1657us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:37:31.839762 (+ 1606us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:37:31.840377 (+ 615us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:37:31.840551 (+ 174us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:37:31.850133 (+ 9582us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:37:31.850147 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:37:31.850194 (+ 47us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 18:37:31.850240 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:37:31.853252 (+ 3012us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:37:31.853910 (+ 658us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:37:31.853917 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:37:31.853922 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:37:31.854065 (+ 143us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:37:31.854478 (+ 413us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:37:31.854481 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:37:31.854483 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:37:31.854951 (+ 468us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:37:31.855067 (+ 116us) server_negotiation.cc:1035] Waiting for connection context
0806 18:37:31.855806 (+ 739us) server_negotiation.cc:299] Negotiation successful
0806 18:37:31.856128 (+ 322us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":240,"thread_start_us":114,"threads_started":1}
I20260806 18:37:31.857730 4468 ts_tablet_manager.cc:1428] T 77eea4fef45b4b2fb4976e06fb15f386 P 5fdb1c79f09d42078b9fadc4624e6d0d: Time spent starting tablet: real 0.006s user 0.006s sys 0.000s
I20260806 18:37:31.858515 4585 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:37:31.831728 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.194.62:36421 (local address 127.19.194.1:60547)
0806 18:37:31.832381 (+ 653us) negotiation.cc:107] Waiting for socket to connect
0806 18:37:31.832471 (+ 90us) client_negotiation.cc:174] Beginning negotiation
0806 18:37:31.833601 (+ 1130us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:37:31.835079 (+ 1478us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:37:31.835091 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:37:31.835647 (+ 556us) client_negotiation.cc:189] Negotiated authn=SASL
0806 18:37:31.836293 (+ 646us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:37:31.836315 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:37:31.838342 (+ 2027us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:37:31.838345 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:37:31.839619 (+ 1274us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:37:31.839629 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:37:31.840536 (+ 907us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:37:31.840540 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:37:31.840726 (+ 186us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:37:31.841345 (+ 619us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 18:37:31.841360 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0806 18:37:31.843328 (+ 1968us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 18:37:31.853404 (+ 10076us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:37:31.853409 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:37:31.853422 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:37:31.853763 (+ 341us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:37:31.854194 (+ 431us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:37:31.854197 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:37:31.854198 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:37:31.854372 (+ 174us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:37:31.855158 (+ 786us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 18:37:31.855164 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 18:37:31.855447 (+ 283us) client_negotiation.cc:769] Sending connection context
0806 18:37:31.857396 (+ 1949us) client_negotiation.cc:240] Negotiation successful
0806 18:37:31.857713 (+ 317us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":348,"thread_start_us":167,"threads_started":1}
I20260806 18:37:31.861351 4583 heartbeater.cc:347] Connected to a master server at 127.19.194.62:36421
I20260806 18:37:31.861618 4583 heartbeater.cc:464] Registering TS with master...
I20260806 18:37:31.862092 4583 heartbeater.cc:511] Master 127.19.194.62:36421 requested a full tablet report, sending...
I20260806 18:37:31.864382 4380 ts_manager.cc:194] Registered new tserver with Master: 5fdb1c79f09d42078b9fadc4624e6d0d (127.19.194.1:36577)
I20260806 18:37:31.868005 4380 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.19.194.1@KRBTEST.COM'} at 127.19.194.1:60547
I20260806 18:37:31.870654 4583 heartbeater.cc:503] Master 127.19.194.62:36421 was elected leader, sending a full tablet report...
Aug 06 18:37:31 dist-test-slave-1xrh krb5kdc[3760](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041451, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.194.62@KRBTEST.COM for kudu/127.19.194.1@KRBTEST.COM
I20260806 18:37:31.878345 4592 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:37:31.867356 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.194.1:36577 (local address 127.0.0.1:40424)
0806 18:37:31.867755 (+ 399us) negotiation.cc:107] Waiting for socket to connect
0806 18:37:31.867798 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0806 18:37:31.867920 (+ 122us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:37:31.868880 (+ 960us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:37:31.868886 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:37:31.869251 (+ 365us) client_negotiation.cc:189] Negotiated authn=SASL
0806 18:37:31.869352 (+ 101us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:37:31.869358 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:37:31.870702 (+ 1344us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:37:31.870707 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:37:31.871737 (+ 1030us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:37:31.871748 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:37:31.872637 (+ 889us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:37:31.872639 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:37:31.872718 (+ 79us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:37:31.873243 (+ 525us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 18:37:31.873254 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0806 18:37:31.874723 (+ 1469us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 18:37:31.876815 (+ 2092us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:37:31.876823 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:37:31.876827 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:37:31.877046 (+ 219us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:37:31.877343 (+ 297us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:37:31.877345 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:37:31.877346 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:37:31.877426 (+ 80us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:37:31.877880 (+ 454us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 18:37:31.877884 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 18:37:31.877986 (+ 102us) client_negotiation.cc:769] Sending connection context
0806 18:37:31.878093 (+ 107us) client_negotiation.cc:240] Negotiation successful
0806 18:37:31.878228 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":233,"thread_start_us":118,"threads_started":1}
I20260806 18:37:31.878700 4593 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:37:31.868075 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40424 (local address 127.19.194.1:36577)
0806 18:37:31.868395 (+ 320us) server_negotiation.cc:206] Beginning negotiation
0806 18:37:31.868415 (+ 20us) server_negotiation.cc:399] Waiting for connection header
0806 18:37:31.868438 (+ 23us) server_negotiation.cc:407] Connection header received
0806 18:37:31.868499 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:37:31.868504 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:37:31.868695 (+ 191us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:37:31.869491 (+ 796us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:37:31.869828 (+ 337us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:37:31.870586 (+ 758us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:37:31.871873 (+ 1287us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:37:31.872521 (+ 648us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:37:31.872631 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:37:31.874863 (+ 2232us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:37:31.874873 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:37:31.874883 (+ 10us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 18:37:31.874925 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:37:31.876696 (+ 1771us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:37:31.877153 (+ 457us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:37:31.877159 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:37:31.877161 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:37:31.877252 (+ 91us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:37:31.877513 (+ 261us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:37:31.877516 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:37:31.877518 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:37:31.877769 (+ 251us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:37:31.877912 (+ 143us) server_negotiation.cc:1035] Waiting for connection context
0806 18:37:31.878206 (+ 294us) server_negotiation.cc:299] Negotiation successful
0806 18:37:31.878344 (+ 138us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":227,"thread_start_us":121,"threads_started":1}
I20260806 18:37:31.879612 4512 tablet_service.cc:1514] Processing DeleteTablet for tablet 77eea4fef45b4b2fb4976e06fb15f386 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 18:37:11 UTC) from {username='kudu', principal='kudu/127.19.194.62@KRBTEST.COM'} at 127.0.0.1:40424
I20260806 18:37:31.880445 4594 tablet_replica.cc:330] stopping tablet replica
I20260806 18:37:31.880735 4594 raft_consensus.cc:2229] T 77eea4fef45b4b2fb4976e06fb15f386 P 5fdb1c79f09d42078b9fadc4624e6d0d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 18:37:31.880988 4594 raft_consensus.cc:2258] T 77eea4fef45b4b2fb4976e06fb15f386 P 5fdb1c79f09d42078b9fadc4624e6d0d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 18:37:31.881992 4594 ts_tablet_manager.cc:1905] T 77eea4fef45b4b2fb4976e06fb15f386 P 5fdb1c79f09d42078b9fadc4624e6d0d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 18:37:31.886354 4594 ts_tablet_manager.cc:1918] T 77eea4fef45b4b2fb4976e06fb15f386 P 5fdb1c79f09d42078b9fadc4624e6d0d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 18:37:31.886452 4594 log.cc:1196] T 77eea4fef45b4b2fb4976e06fb15f386 P 5fdb1c79f09d42078b9fadc4624e6d0d: Deleting WAL directory at /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786041246094603-20232-0/minicluster-data/ts-0/wal/wals/77eea4fef45b4b2fb4976e06fb15f386
I20260806 18:37:31.886734 4594 ts_tablet_manager.cc:1939] T 77eea4fef45b4b2fb4976e06fb15f386 P 5fdb1c79f09d42078b9fadc4624e6d0d: Deleting consensus metadata
I20260806 18:37:31.888473 4362 catalog_manager.cc:4841] TS 5fdb1c79f09d42078b9fadc4624e6d0d (127.19.194.1:36577): tablet 77eea4fef45b4b2fb4976e06fb15f386 (table test-table [id=cd76f96f476143f6a38dddbaef9c6ea8]) successfully deleted
W20260806 18:37:31.960585 4589 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 18:37:31.960822 4589 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 18:37:31.960865 4589 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 18:37:31.964779 4589 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 18:37:31.964829 4589 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 18:37:31.964884 4589 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.19.194.2
I20260806 18:37:31.969944 4589 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.194.20:36777
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786041246094603-20232-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786041246094603-20232-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786041246094603-20232-0/krb5kdc/kudu_127.19.194.2.keytab
--principal=kudu/127.19.194.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.19.194.2:46303
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786041246094603-20232-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.19.194.2
--webserver_port=38271
--tserver_master_addrs=127.19.194.62:36421
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4589
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.19.194.2
--log_dir=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786041246094603-20232-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 17:59:13 UTC on ccycloud.quasar-wvsnzx.root.comops.site
build id 4571028
I20260806 18:37:31.970932 4589 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 18:37:31.971922 4589 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 18:37:31.981272 4600 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 18:37:31.981492 4589 server_base.cc:1034] running on GCE node
W20260806 18:37:31.981230 4602 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:37:31.981237 4599 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 18:37:31.981972 4589 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 18:37:31.982632 4589 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 18:37:31.983831 4589 hybrid_clock.cc:648] HybridClock initialized: now 1786041451983807 us; error 37 us; skew 500 ppm
Aug 06 18:37:31 dist-test-slave-1xrh krb5kdc[3760](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041451, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.194.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 18:37:31.986557 4589 init.cc:377] Logged in from keytab as kudu/127.19.194.2@KRBTEST.COM (short username kudu)
I20260806 18:37:31.987476 4589 webserver.cc:460] Webserver started at http://127.19.194.2:38271/ using document root <none> and password file <none>
I20260806 18:37:31.988014 4589 fs_manager.cc:359] Metadata directory not provided
I20260806 18:37:31.988132 4589 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 18:37:31.991950 4589 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.004s sys 0.000s
I20260806 18:37:31.993853 4609 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 18:37:31.995020 4589 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260806 18:37:31.995337 4589 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786041246094603-20232-0/minicluster-data/ts-1/data,/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786041246094603-20232-0/minicluster-data/ts-1/wal
uuid: "4d2a68a0b77a43619f6206aee728280d"
format_stamp: "Formatted at 2026-08-06 18:37:10 on dist-test-slave-1xrh"
server_key: "f9d523d313ee70f29dd5f5475c7ff29c"
server_key_iv: "313833d9c7638513ffd6527402988301"
server_key_version: "encryptionkey@0"
I20260806 18:37:31.995900 4589 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786041246094603-20232-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786041246094603-20232-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786041246094603-20232-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 18:37:32.012264 4589 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 18:37:32.015480 4589 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 18:37:32.015681 4589 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 18:37:32.016490 4589 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 18:37:32.017535 4589 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 18:37:32.017604 4589 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 18:37:32.017670 4589 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 18:37:32.017699 4589 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 18:37:32.026546 4589 rpc_server.cc:307] RPC server started. Bound to: 127.19.194.2:46303
I20260806 18:37:32.026572 4727 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.194.2:46303 every 8 connection(s)
I20260806 18:37:32.027530 4589 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786041246094603-20232-0/minicluster-data/ts-1/data/info.pb
I20260806 18:37:32.032836 20232 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJW69tn/build/debug/bin/kudu as pid 4589
I20260806 18:37:32.033313 20232 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJW69tn/build/debug/bin/kudu
/tmp/dist-test-taskJW69tn/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786041246094603-20232-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786041246094603-20232-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786041246094603-20232-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786041246094603-20232-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.19.194.3:35207
--local_ip_for_outbound_sockets=127.19.194.3
--tserver_master_addrs=127.19.194.62:36421
--webserver_port=33259
--webserver_interface=127.19.194.3
--keytab_file=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786041246094603-20232-0/krb5kdc/kudu_127.19.194.3.keytab
--principal=kudu/127.19.194.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.19.194.20:36777
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786041246094603-20232-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786041246094603-20232-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786041246094603-20232-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 06 18:37:32 dist-test-slave-1xrh krb5kdc[3760](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041451, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.194.2@KRBTEST.COM for kudu/127.19.194.62@KRBTEST.COM
I20260806 18:37:32.041620 4586 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:37:32.029273 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.194.2:47311 (local address 127.19.194.62:36421)
0806 18:37:32.029421 (+ 148us) server_negotiation.cc:206] Beginning negotiation
0806 18:37:32.029424 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 18:37:32.030156 (+ 732us) server_negotiation.cc:407] Connection header received
0806 18:37:32.031498 (+ 1342us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:37:32.031501 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:37:32.031562 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:37:32.031655 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:37:32.032763 (+ 1108us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:37:32.033428 (+ 665us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:37:32.034554 (+ 1126us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:37:32.034789 (+ 235us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:37:32.034917 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:37:32.037381 (+ 2464us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:37:32.037387 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:37:32.037391 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 18:37:32.037424 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:37:32.039263 (+ 1839us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:37:32.039863 (+ 600us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:37:32.039868 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:37:32.039872 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:37:32.039970 (+ 98us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:37:32.040327 (+ 357us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:37:32.040333 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:37:32.040337 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:37:32.040621 (+ 284us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:37:32.040733 (+ 112us) server_negotiation.cc:1035] Waiting for connection context
0806 18:37:32.041201 (+ 468us) server_negotiation.cc:299] Negotiation successful
0806 18:37:32.041365 (+ 164us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260806 18:37:32.042181 4730 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:37:32.029587 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.194.62:36421 (local address 127.19.194.2:47311)
0806 18:37:32.030007 (+ 420us) negotiation.cc:107] Waiting for socket to connect
0806 18:37:32.030047 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0806 18:37:32.031289 (+ 1242us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:37:32.031817 (+ 528us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:37:32.031823 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:37:32.032132 (+ 309us) client_negotiation.cc:189] Negotiated authn=SASL
0806 18:37:32.032624 (+ 492us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:37:32.032637 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:37:32.033583 (+ 946us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:37:32.033585 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:37:32.034427 (+ 842us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:37:32.034437 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:37:32.034943 (+ 506us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:37:32.034948 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:37:32.035117 (+ 169us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:37:32.035756 (+ 639us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 18:37:32.035771 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0806 18:37:32.037180 (+ 1409us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 18:37:32.039382 (+ 2202us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:37:32.039387 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:37:32.039397 (+ 10us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:37:32.039763 (+ 366us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:37:32.040099 (+ 336us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:37:32.040101 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:37:32.040102 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:37:32.040229 (+ 127us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:37:32.040735 (+ 506us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 18:37:32.040739 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 18:37:32.040965 (+ 226us) client_negotiation.cc:769] Sending connection context
0806 18:37:32.041111 (+ 146us) client_negotiation.cc:240] Negotiation successful
0806 18:37:32.041336 (+ 225us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":249,"thread_start_us":109,"threads_started":1}
I20260806 18:37:32.043637 4728 heartbeater.cc:347] Connected to a master server at 127.19.194.62:36421
I20260806 18:37:32.043891 4728 heartbeater.cc:464] Registering TS with master...
I20260806 18:37:32.044400 4728 heartbeater.cc:511] Master 127.19.194.62:36421 requested a full tablet report, sending...
I20260806 18:37:32.045516 4380 ts_manager.cc:194] Registered new tserver with Master: 4d2a68a0b77a43619f6206aee728280d (127.19.194.2:46303)
I20260806 18:37:32.046314 4380 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.19.194.2@KRBTEST.COM'} at 127.19.194.2:47311
W20260806 18:37:32.144594 4731 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 18:37:32.144852 4731 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 18:37:32.144898 4731 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 18:37:32.149150 4731 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 18:37:32.149194 4731 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 18:37:32.149248 4731 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.19.194.3
I20260806 18:37:32.153863 4731 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.194.20:36777
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786041246094603-20232-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786041246094603-20232-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786041246094603-20232-0/krb5kdc/kudu_127.19.194.3.keytab
--principal=kudu/127.19.194.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.19.194.3:35207
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786041246094603-20232-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.19.194.3
--webserver_port=33259
--tserver_master_addrs=127.19.194.62:36421
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4731
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.19.194.3
--log_dir=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786041246094603-20232-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 17:59:13 UTC on ccycloud.quasar-wvsnzx.root.comops.site
build id 4571028
I20260806 18:37:32.154700 4731 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 18:37:32.155606 4731 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 18:37:32.164449 4738 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:37:32.164423 4737 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:37:32.164438 4740 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 18:37:32.164800 4731 server_base.cc:1034] running on GCE node
I20260806 18:37:32.165190 4731 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 18:37:32.165925 4731 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 18:37:32.167090 4731 hybrid_clock.cc:648] HybridClock initialized: now 1786041452167073 us; error 34 us; skew 500 ppm
Aug 06 18:37:32 dist-test-slave-1xrh krb5kdc[3760](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041452, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.194.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 18:37:32.169718 4731 init.cc:377] Logged in from keytab as kudu/127.19.194.3@KRBTEST.COM (short username kudu)
I20260806 18:37:32.170504 4731 webserver.cc:460] Webserver started at http://127.19.194.3:33259/ using document root <none> and password file <none>
I20260806 18:37:32.171000 4731 fs_manager.cc:359] Metadata directory not provided
I20260806 18:37:32.171093 4731 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 18:37:32.174672 4731 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.000s sys 0.002s
I20260806 18:37:32.176671 4747 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 18:37:32.177739 4731 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260806 18:37:32.177865 4731 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786041246094603-20232-0/minicluster-data/ts-2/data,/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786041246094603-20232-0/minicluster-data/ts-2/wal
uuid: "c5a13411dadf4739ba09a5da11f16926"
format_stamp: "Formatted at 2026-08-06 18:37:11 on dist-test-slave-1xrh"
server_key: "702a906614a2f7717547b4cb83b505a1"
server_key_iv: "8fafdc8e52d92aa949a83dcb793bd92c"
server_key_version: "encryptionkey@0"
I20260806 18:37:32.178380 4731 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786041246094603-20232-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786041246094603-20232-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786041246094603-20232-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 18:37:32.203917 4731 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 18:37:32.206898 4731 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 18:37:32.207101 4731 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 18:37:32.207687 4731 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 18:37:32.208473 4731 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 18:37:32.208541 4731 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 18:37:32.208606 4731 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 18:37:32.208657 4731 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 18:37:32.217211 4731 rpc_server.cc:307] RPC server started. Bound to: 127.19.194.3:35207
I20260806 18:37:32.217257 4867 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.194.3:35207 every 8 connection(s)
I20260806 18:37:32.218122 4731 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786041246094603-20232-0/minicluster-data/ts-2/data/info.pb
I20260806 18:37:32.220170 20232 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJW69tn/build/debug/bin/kudu as pid 4731
Aug 06 18:37:32 dist-test-slave-1xrh krb5kdc[3760](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041452, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.194.3@KRBTEST.COM for kudu/127.19.194.62@KRBTEST.COM
I20260806 18:37:32.229058 4872 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:37:32.220872 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41794 (local address 127.19.194.62:36421)
0806 18:37:32.221135 (+ 263us) server_negotiation.cc:206] Beginning negotiation
0806 18:37:32.221139 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 18:37:32.221252 (+ 113us) server_negotiation.cc:407] Connection header received
0806 18:37:32.221351 (+ 99us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:37:32.221354 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:37:32.221413 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:37:32.221533 (+ 120us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:37:32.222240 (+ 707us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:37:32.223038 (+ 798us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:37:32.224020 (+ 982us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:37:32.224251 (+ 231us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:37:32.224329 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:37:32.225709 (+ 1380us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:37:32.225713 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:37:32.225716 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 18:37:32.225746 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:37:32.227400 (+ 1654us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:37:32.227796 (+ 396us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:37:32.227800 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:37:32.227802 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:37:32.227889 (+ 87us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:37:32.228187 (+ 298us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:37:32.228191 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:37:32.228193 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:37:32.228415 (+ 222us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:37:32.228570 (+ 155us) server_negotiation.cc:1035] Waiting for connection context
0806 18:37:32.228784 (+ 214us) server_negotiation.cc:299] Negotiation successful
0806 18:37:32.228883 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":2,"server-negotiator.queue_time_us":199,"thread_start_us":106,"threads_started":1}
I20260806 18:37:32.232558 4586 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:37:32.219776 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.194.3:33803 (local address 127.19.194.62:36421)
0806 18:37:32.219960 (+ 184us) server_negotiation.cc:206] Beginning negotiation
0806 18:37:32.219966 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 18:37:32.220825 (+ 859us) server_negotiation.cc:407] Connection header received
0806 18:37:32.222493 (+ 1668us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:37:32.222496 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:37:32.222535 (+ 39us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:37:32.222594 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:37:32.223855 (+ 1261us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:37:32.224469 (+ 614us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:37:32.225725 (+ 1256us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:37:32.225928 (+ 203us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:37:32.226050 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:37:32.228946 (+ 2896us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:37:32.228950 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:37:32.228952 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 18:37:32.228982 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:37:32.230484 (+ 1502us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:37:32.231070 (+ 586us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:37:32.231074 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:37:32.231076 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:37:32.231236 (+ 160us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:37:32.231579 (+ 343us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:37:32.231583 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:37:32.231585 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:37:32.231747 (+ 162us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:37:32.231870 (+ 123us) server_negotiation.cc:1035] Waiting for connection context
0806 18:37:32.232330 (+ 460us) server_negotiation.cc:299] Negotiation successful
0806 18:37:32.232434 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260806 18:37:32.233281 4870 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:37:32.220153 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.194.62:36421 (local address 127.19.194.3:33803)
0806 18:37:32.220664 (+ 511us) negotiation.cc:107] Waiting for socket to connect
0806 18:37:32.220724 (+ 60us) client_negotiation.cc:174] Beginning negotiation
0806 18:37:32.222282 (+ 1558us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:37:32.222764 (+ 482us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:37:32.222772 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:37:32.223246 (+ 474us) client_negotiation.cc:189] Negotiated authn=SASL
0806 18:37:32.223681 (+ 435us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:37:32.223694 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:37:32.224597 (+ 903us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:37:32.224599 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:37:32.225569 (+ 970us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:37:32.225578 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:37:32.226108 (+ 530us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:37:32.226114 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:37:32.226327 (+ 213us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:37:32.226860 (+ 533us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 18:37:32.226881 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0806 18:37:32.228768 (+ 1887us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 18:37:32.230597 (+ 1829us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:37:32.230602 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:37:32.230615 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:37:32.230969 (+ 354us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:37:32.231338 (+ 369us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:37:32.231342 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:37:32.231344 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:37:32.231481 (+ 137us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:37:32.231926 (+ 445us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 18:37:32.231931 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 18:37:32.232125 (+ 194us) client_negotiation.cc:769] Sending connection context
0806 18:37:32.232268 (+ 143us) client_negotiation.cc:240] Negotiation successful
0806 18:37:32.232503 (+ 235us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":309,"thread_start_us":149,"threads_started":1}
I20260806 18:37:32.234232 4868 heartbeater.cc:347] Connected to a master server at 127.19.194.62:36421
I20260806 18:37:32.234452 4868 heartbeater.cc:464] Registering TS with master...
I20260806 18:37:32.234894 4868 heartbeater.cc:511] Master 127.19.194.62:36421 requested a full tablet report, sending...
I20260806 18:37:32.235846 4380 ts_manager.cc:194] Registered new tserver with Master: c5a13411dadf4739ba09a5da11f16926 (127.19.194.3:35207)
I20260806 18:37:32.236368 4380 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.19.194.3@KRBTEST.COM'} at 127.19.194.3:33803
I20260806 18:37:32.242278 20232 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260806 18:37:32.246153 4586 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:37:32.242801 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41804 (local address 127.19.194.62:36421)
0806 18:37:32.242937 (+ 136us) server_negotiation.cc:206] Beginning negotiation
0806 18:37:32.242941 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 18:37:32.243118 (+ 177us) server_negotiation.cc:407] Connection header received
0806 18:37:32.243298 (+ 180us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:37:32.243301 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:37:32.243394 (+ 93us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:37:32.243510 (+ 116us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 18:37:32.243736 (+ 226us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:37:32.244235 (+ 499us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:37:32.245096 (+ 861us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:37:32.245312 (+ 216us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:37:32.245412 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:37:32.245692 (+ 280us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 18:37:32.245807 (+ 115us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_AUTHENTICATION_TOKEN: Not authorized: token expired
0806 18:37:32.246035 (+ 228us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:41804: token expired
Metrics: {"server-negotiator.queue_time_us":47}
W20260806 18:37:32.246219 4586 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:41804: token expired
W20260806 18:37:32.246528 4297 master_proxy_rpc.cc:203] Re-attempting CreateTable request to leader Master (127.19.194.62:36421)
I20260806 18:37:32.253597 4586 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:37:32.246874 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41810 (local address 127.19.194.62:36421)
0806 18:37:32.246989 (+ 115us) server_negotiation.cc:206] Beginning negotiation
0806 18:37:32.246995 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 18:37:32.247036 (+ 41us) server_negotiation.cc:407] Connection header received
0806 18:37:32.247201 (+ 165us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:37:32.247204 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:37:32.247241 (+ 37us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:37:32.247325 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:37:32.247915 (+ 590us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:37:32.248386 (+ 471us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:37:32.249163 (+ 777us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:37:32.249385 (+ 222us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:37:32.249496 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:37:32.250724 (+ 1228us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:37:32.250728 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:37:32.250730 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 18:37:32.250753 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:37:32.252027 (+ 1274us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:37:32.252443 (+ 416us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:37:32.252447 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:37:32.252449 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:37:32.252533 (+ 84us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:37:32.252795 (+ 262us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:37:32.252799 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:37:32.252800 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:37:32.252959 (+ 159us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:37:32.253093 (+ 134us) server_negotiation.cc:1035] Waiting for connection context
0806 18:37:32.253324 (+ 231us) server_negotiation.cc:299] Negotiation successful
0806 18:37:32.253424 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260806 18:37:32.255800 4380 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:41810:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260806 18:37:32.258927 4380 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 18:37:32.266546 4512 tablet_service.cc:1467] Processing CreateTablet for tablet 41b7a086ba4043ecb933bd4721a649ba (DEFAULT_TABLE table=test-table [id=73c0eab94a454780b9b697943bae4e77]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 18:37:32.267234 4512 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 41b7a086ba4043ecb933bd4721a649ba. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 18:37:32.269950 4468 tablet_bootstrap.cc:492] T 41b7a086ba4043ecb933bd4721a649ba P 5fdb1c79f09d42078b9fadc4624e6d0d: Bootstrap starting.
I20260806 18:37:32.270696 4592 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:37:32.265369 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.194.3:35207 (local address 127.0.0.1:45974)
0806 18:37:32.265473 (+ 104us) negotiation.cc:107] Waiting for socket to connect
0806 18:37:32.265490 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0806 18:37:32.265565 (+ 75us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:37:32.266613 (+ 1048us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:37:32.266616 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:37:32.266637 (+ 21us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 18:37:32.266723 (+ 86us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:37:32.266728 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:37:32.268145 (+ 1417us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:37:32.268148 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:37:32.269330 (+ 1182us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:37:32.269339 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:37:32.270264 (+ 925us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:37:32.270270 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:37:32.270374 (+ 104us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:37:32.270422 (+ 48us) client_negotiation.cc:769] Sending connection context
0806 18:37:32.270507 (+ 85us) client_negotiation.cc:240] Negotiation successful
0806 18:37:32.270579 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":34}
I20260806 18:37:32.270846 4468 tablet_bootstrap.cc:654] T 41b7a086ba4043ecb933bd4721a649ba P 5fdb1c79f09d42078b9fadc4624e6d0d: Neither blocks nor log segments found. Creating new log.
I20260806 18:37:32.271412 4878 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:37:32.265601 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45974 (local address 127.19.194.3:35207)
0806 18:37:32.266143 (+ 542us) server_negotiation.cc:206] Beginning negotiation
0806 18:37:32.266158 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0806 18:37:32.266181 (+ 23us) server_negotiation.cc:407] Connection header received
0806 18:37:32.266252 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:37:32.266257 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:37:32.266448 (+ 191us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:37:32.266616 (+ 168us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 18:37:32.266833 (+ 217us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:37:32.268003 (+ 1170us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:37:32.269453 (+ 1450us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:37:32.270142 (+ 689us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:37:32.270251 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:37:32.270388 (+ 137us) server_negotiation.cc:1035] Waiting for connection context
0806 18:37:32.271158 (+ 770us) server_negotiation.cc:299] Negotiation successful
0806 18:37:32.271256 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":430,"thread_start_us":185,"threads_started":1}
I20260806 18:37:32.271723 4877 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:37:32.265687 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.194.2:46303 (local address 127.0.0.1:41602)
0806 18:37:32.266315 (+ 628us) negotiation.cc:107] Waiting for socket to connect
0806 18:37:32.266325 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0806 18:37:32.266392 (+ 67us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:37:32.267354 (+ 962us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:37:32.267358 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:37:32.267378 (+ 20us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 18:37:32.267471 (+ 93us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:37:32.267478 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:37:32.269183 (+ 1705us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:37:32.269187 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:37:32.270229 (+ 1042us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:37:32.270242 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:37:32.271361 (+ 1119us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:37:32.271365 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:37:32.271458 (+ 93us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:37:32.271490 (+ 32us) client_negotiation.cc:769] Sending connection context
0806 18:37:32.271547 (+ 57us) client_negotiation.cc:240] Negotiation successful
0806 18:37:32.271607 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":577,"thread_start_us":85,"threads_started":1}
I20260806 18:37:32.272419 4880 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:37:32.265969 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41602 (local address 127.19.194.2:46303)
0806 18:37:32.266639 (+ 670us) server_negotiation.cc:206] Beginning negotiation
0806 18:37:32.266650 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0806 18:37:32.266671 (+ 21us) server_negotiation.cc:407] Connection header received
0806 18:37:32.266740 (+ 69us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:37:32.266764 (+ 24us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:37:32.267038 (+ 274us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:37:32.267623 (+ 585us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 18:37:32.267937 (+ 314us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:37:32.269051 (+ 1114us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:37:32.270365 (+ 1314us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:37:32.271227 (+ 862us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:37:32.271943 (+ 716us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:37:32.272104 (+ 161us) server_negotiation.cc:1035] Waiting for connection context
0806 18:37:32.272156 (+ 52us) server_negotiation.cc:299] Negotiation successful
0806 18:37:32.272251 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":553,"thread_start_us":103,"threads_started":1}
I20260806 18:37:32.272702 4468 tablet_bootstrap.cc:492] T 41b7a086ba4043ecb933bd4721a649ba P 5fdb1c79f09d42078b9fadc4624e6d0d: No bootstrap required, opened a new log
I20260806 18:37:32.272804 4468 ts_tablet_manager.cc:1397] T 41b7a086ba4043ecb933bd4721a649ba P 5fdb1c79f09d42078b9fadc4624e6d0d: Time spent bootstrapping tablet: real 0.003s user 0.000s sys 0.001s
I20260806 18:37:32.273209 4468 raft_consensus.cc:348] T 41b7a086ba4043ecb933bd4721a649ba P 5fdb1c79f09d42078b9fadc4624e6d0d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c5a13411dadf4739ba09a5da11f16926" member_type: VOTER last_known_addr { host: "127.19.194.3" port: 35207 } } peers { permanent_uuid: "4d2a68a0b77a43619f6206aee728280d" member_type: VOTER last_known_addr { host: "127.19.194.2" port: 46303 } } peers { permanent_uuid: "5fdb1c79f09d42078b9fadc4624e6d0d" member_type: VOTER last_known_addr { host: "127.19.194.1" port: 36577 } }
I20260806 18:37:32.273350 4468 raft_consensus.cc:374] T 41b7a086ba4043ecb933bd4721a649ba P 5fdb1c79f09d42078b9fadc4624e6d0d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 18:37:32.273411 4468 raft_consensus.cc:729] T 41b7a086ba4043ecb933bd4721a649ba P 5fdb1c79f09d42078b9fadc4624e6d0d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5fdb1c79f09d42078b9fadc4624e6d0d, State: Initialized, Role: FOLLOWER
I20260806 18:37:32.273536 4468 consensus_queue.cc:260] T 41b7a086ba4043ecb933bd4721a649ba P 5fdb1c79f09d42078b9fadc4624e6d0d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c5a13411dadf4739ba09a5da11f16926" member_type: VOTER last_known_addr { host: "127.19.194.3" port: 35207 } } peers { permanent_uuid: "4d2a68a0b77a43619f6206aee728280d" member_type: VOTER last_known_addr { host: "127.19.194.2" port: 46303 } } peers { permanent_uuid: "5fdb1c79f09d42078b9fadc4624e6d0d" member_type: VOTER last_known_addr { host: "127.19.194.1" port: 36577 } }
I20260806 18:37:32.273903 4468 ts_tablet_manager.cc:1428] T 41b7a086ba4043ecb933bd4721a649ba P 5fdb1c79f09d42078b9fadc4624e6d0d: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20260806 18:37:32.274629 4796 tablet_service.cc:1467] Processing CreateTablet for tablet 41b7a086ba4043ecb933bd4721a649ba (DEFAULT_TABLE table=test-table [id=73c0eab94a454780b9b697943bae4e77]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 18:37:32.275055 4657 tablet_service.cc:1467] Processing CreateTablet for tablet 41b7a086ba4043ecb933bd4721a649ba (DEFAULT_TABLE table=test-table [id=73c0eab94a454780b9b697943bae4e77]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 18:37:32.275765 4796 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 41b7a086ba4043ecb933bd4721a649ba. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 18:37:32.276121 4657 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 41b7a086ba4043ecb933bd4721a649ba. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 18:37:32.282702 4882 tablet_bootstrap.cc:492] T 41b7a086ba4043ecb933bd4721a649ba P c5a13411dadf4739ba09a5da11f16926: Bootstrap starting.
I20260806 18:37:32.282896 4883 tablet_bootstrap.cc:492] T 41b7a086ba4043ecb933bd4721a649ba P 4d2a68a0b77a43619f6206aee728280d: Bootstrap starting.
I20260806 18:37:32.285243 4882 tablet_bootstrap.cc:654] T 41b7a086ba4043ecb933bd4721a649ba P c5a13411dadf4739ba09a5da11f16926: Neither blocks nor log segments found. Creating new log.
I20260806 18:37:32.285277 4883 tablet_bootstrap.cc:654] T 41b7a086ba4043ecb933bd4721a649ba P 4d2a68a0b77a43619f6206aee728280d: Neither blocks nor log segments found. Creating new log.
I20260806 18:37:32.285960 4882 log.cc:824] T 41b7a086ba4043ecb933bd4721a649ba P c5a13411dadf4739ba09a5da11f16926: Log is configured to *not* fsync() on all Append() calls
I20260806 18:37:32.285969 4883 log.cc:824] T 41b7a086ba4043ecb933bd4721a649ba P 4d2a68a0b77a43619f6206aee728280d: Log is configured to *not* fsync() on all Append() calls
I20260806 18:37:32.287752 4883 tablet_bootstrap.cc:492] T 41b7a086ba4043ecb933bd4721a649ba P 4d2a68a0b77a43619f6206aee728280d: No bootstrap required, opened a new log
I20260806 18:37:32.287791 4882 tablet_bootstrap.cc:492] T 41b7a086ba4043ecb933bd4721a649ba P c5a13411dadf4739ba09a5da11f16926: No bootstrap required, opened a new log
I20260806 18:37:32.287987 4882 ts_tablet_manager.cc:1397] T 41b7a086ba4043ecb933bd4721a649ba P c5a13411dadf4739ba09a5da11f16926: Time spent bootstrapping tablet: real 0.005s user 0.004s sys 0.000s
I20260806 18:37:32.287992 4883 ts_tablet_manager.cc:1397] T 41b7a086ba4043ecb933bd4721a649ba P 4d2a68a0b77a43619f6206aee728280d: Time spent bootstrapping tablet: real 0.005s user 0.004s sys 0.000s
I20260806 18:37:32.291047 4882 raft_consensus.cc:348] T 41b7a086ba4043ecb933bd4721a649ba P c5a13411dadf4739ba09a5da11f16926 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c5a13411dadf4739ba09a5da11f16926" member_type: VOTER last_known_addr { host: "127.19.194.3" port: 35207 } } peers { permanent_uuid: "4d2a68a0b77a43619f6206aee728280d" member_type: VOTER last_known_addr { host: "127.19.194.2" port: 46303 } } peers { permanent_uuid: "5fdb1c79f09d42078b9fadc4624e6d0d" member_type: VOTER last_known_addr { host: "127.19.194.1" port: 36577 } }
I20260806 18:37:32.291090 4883 raft_consensus.cc:348] T 41b7a086ba4043ecb933bd4721a649ba P 4d2a68a0b77a43619f6206aee728280d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c5a13411dadf4739ba09a5da11f16926" member_type: VOTER last_known_addr { host: "127.19.194.3" port: 35207 } } peers { permanent_uuid: "4d2a68a0b77a43619f6206aee728280d" member_type: VOTER last_known_addr { host: "127.19.194.2" port: 46303 } } peers { permanent_uuid: "5fdb1c79f09d42078b9fadc4624e6d0d" member_type: VOTER last_known_addr { host: "127.19.194.1" port: 36577 } }
I20260806 18:37:32.291355 4882 raft_consensus.cc:374] T 41b7a086ba4043ecb933bd4721a649ba P c5a13411dadf4739ba09a5da11f16926 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 18:37:32.291427 4882 raft_consensus.cc:729] T 41b7a086ba4043ecb933bd4721a649ba P c5a13411dadf4739ba09a5da11f16926 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c5a13411dadf4739ba09a5da11f16926, State: Initialized, Role: FOLLOWER
I20260806 18:37:32.291424 4883 raft_consensus.cc:374] T 41b7a086ba4043ecb933bd4721a649ba P 4d2a68a0b77a43619f6206aee728280d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 18:37:32.291481 4883 raft_consensus.cc:729] T 41b7a086ba4043ecb933bd4721a649ba P 4d2a68a0b77a43619f6206aee728280d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4d2a68a0b77a43619f6206aee728280d, State: Initialized, Role: FOLLOWER
I20260806 18:37:32.291828 4882 consensus_queue.cc:260] T 41b7a086ba4043ecb933bd4721a649ba P c5a13411dadf4739ba09a5da11f16926 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c5a13411dadf4739ba09a5da11f16926" member_type: VOTER last_known_addr { host: "127.19.194.3" port: 35207 } } peers { permanent_uuid: "4d2a68a0b77a43619f6206aee728280d" member_type: VOTER last_known_addr { host: "127.19.194.2" port: 46303 } } peers { permanent_uuid: "5fdb1c79f09d42078b9fadc4624e6d0d" member_type: VOTER last_known_addr { host: "127.19.194.1" port: 36577 } }
I20260806 18:37:32.291903 4883 consensus_queue.cc:260] T 41b7a086ba4043ecb933bd4721a649ba P 4d2a68a0b77a43619f6206aee728280d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c5a13411dadf4739ba09a5da11f16926" member_type: VOTER last_known_addr { host: "127.19.194.3" port: 35207 } } peers { permanent_uuid: "4d2a68a0b77a43619f6206aee728280d" member_type: VOTER last_known_addr { host: "127.19.194.2" port: 46303 } } peers { permanent_uuid: "5fdb1c79f09d42078b9fadc4624e6d0d" member_type: VOTER last_known_addr { host: "127.19.194.1" port: 36577 } }
I20260806 18:37:32.292460 4868 heartbeater.cc:503] Master 127.19.194.62:36421 was elected leader, sending a full tablet report...
I20260806 18:37:32.292578 4728 heartbeater.cc:503] Master 127.19.194.62:36421 was elected leader, sending a full tablet report...
I20260806 18:37:32.292807 4882 ts_tablet_manager.cc:1428] T 41b7a086ba4043ecb933bd4721a649ba P c5a13411dadf4739ba09a5da11f16926: Time spent starting tablet: real 0.005s user 0.005s sys 0.000s
I20260806 18:37:32.292865 4883 ts_tablet_manager.cc:1428] T 41b7a086ba4043ecb933bd4721a649ba P 4d2a68a0b77a43619f6206aee728280d: Time spent starting tablet: real 0.005s user 0.005s sys 0.000s
W20260806 18:37:32.331033 4584 tablet.cc:2369] T 41b7a086ba4043ecb933bd4721a649ba P 5fdb1c79f09d42078b9fadc4624e6d0d: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 18:37:32.403669 4888 raft_consensus.cc:482] T 41b7a086ba4043ecb933bd4721a649ba P 4d2a68a0b77a43619f6206aee728280d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 18:37:32.403867 4888 raft_consensus.cc:504] T 41b7a086ba4043ecb933bd4721a649ba P 4d2a68a0b77a43619f6206aee728280d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c5a13411dadf4739ba09a5da11f16926" member_type: VOTER last_known_addr { host: "127.19.194.3" port: 35207 } } peers { permanent_uuid: "4d2a68a0b77a43619f6206aee728280d" member_type: VOTER last_known_addr { host: "127.19.194.2" port: 46303 } } peers { permanent_uuid: "5fdb1c79f09d42078b9fadc4624e6d0d" member_type: VOTER last_known_addr { host: "127.19.194.1" port: 36577 } }
I20260806 18:37:32.405112 4888 leader_election.cc:288] T 41b7a086ba4043ecb933bd4721a649ba P 4d2a68a0b77a43619f6206aee728280d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c5a13411dadf4739ba09a5da11f16926 (127.19.194.3:35207), 5fdb1c79f09d42078b9fadc4624e6d0d (127.19.194.1:36577)
I20260806 18:37:32.410499 4878 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:37:32.405447 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.194.2:46705 (local address 127.19.194.3:35207)
0806 18:37:32.405666 (+ 219us) server_negotiation.cc:206] Beginning negotiation
0806 18:37:32.405671 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 18:37:32.405923 (+ 252us) server_negotiation.cc:407] Connection header received
0806 18:37:32.406046 (+ 123us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:37:32.406049 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:37:32.406100 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:37:32.406189 (+ 89us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 18:37:32.406531 (+ 342us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:37:32.407348 (+ 817us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:37:32.409119 (+ 1771us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:37:32.409720 (+ 601us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:37:32.409858 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:37:32.409911 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0806 18:37:32.410293 (+ 382us) server_negotiation.cc:299] Negotiation successful
0806 18:37:32.410363 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
I20260806 18:37:32.410502 4891 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:37:32.405672 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.194.2:42243 (local address 127.19.194.1:36577)
0806 18:37:32.405978 (+ 306us) server_negotiation.cc:206] Beginning negotiation
0806 18:37:32.405985 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 18:37:32.406000 (+ 15us) server_negotiation.cc:407] Connection header received
0806 18:37:32.406048 (+ 48us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:37:32.406051 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:37:32.406117 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:37:32.406224 (+ 107us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 18:37:32.406531 (+ 307us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:37:32.407362 (+ 831us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:37:32.409119 (+ 1757us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:37:32.409766 (+ 647us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:37:32.409880 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:37:32.410059 (+ 179us) server_negotiation.cc:1035] Waiting for connection context
0806 18:37:32.410293 (+ 234us) server_negotiation.cc:299] Negotiation successful
0806 18:37:32.410371 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":187,"thread_start_us":102,"threads_started":1}
I20260806 18:37:32.410553 4730 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:37:32.405331 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.194.3:35207 (local address 127.19.194.2:46705)
0806 18:37:32.405484 (+ 153us) negotiation.cc:107] Waiting for socket to connect
0806 18:37:32.405502 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0806 18:37:32.405939 (+ 437us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:37:32.406226 (+ 287us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:37:32.406228 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:37:32.406250 (+ 22us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 18:37:32.406405 (+ 155us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:37:32.406413 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:37:32.407468 (+ 1055us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:37:32.407471 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:37:32.408982 (+ 1511us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:37:32.408995 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:37:32.409877 (+ 882us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:37:32.409880 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:37:32.410115 (+ 235us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:37:32.410176 (+ 61us) client_negotiation.cc:769] Sending connection context
0806 18:37:32.410256 (+ 80us) client_negotiation.cc:240] Negotiation successful
0806 18:37:32.410374 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":66,"mutex_wait_us":257,"spinlock_wait_cycles":12544}
I20260806 18:37:32.410562 4890 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:37:32.405359 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.194.1:36577 (local address 127.19.194.2:42243)
0806 18:37:32.405819 (+ 460us) negotiation.cc:107] Waiting for socket to connect
0806 18:37:32.405834 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0806 18:37:32.405935 (+ 101us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:37:32.406237 (+ 302us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:37:32.406239 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:37:32.406285 (+ 46us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 18:37:32.406420 (+ 135us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:37:32.406425 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:37:32.407468 (+ 1043us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:37:32.407471 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:37:32.408982 (+ 1511us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:37:32.408995 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:37:32.409877 (+ 882us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:37:32.409880 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:37:32.410078 (+ 198us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:37:32.410182 (+ 104us) client_negotiation.cc:769] Sending connection context
0806 18:37:32.410256 (+ 74us) client_negotiation.cc:240] Negotiation successful
0806 18:37:32.410374 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":385,"mutex_wait_us":233,"spinlock_wait_cycles":38784,"thread_start_us":102,"threads_started":1}
I20260806 18:37:32.411185 4532 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "41b7a086ba4043ecb933bd4721a649ba" candidate_uuid: "4d2a68a0b77a43619f6206aee728280d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5fdb1c79f09d42078b9fadc4624e6d0d" is_pre_election: true
I20260806 18:37:32.411242 4818 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "41b7a086ba4043ecb933bd4721a649ba" candidate_uuid: "4d2a68a0b77a43619f6206aee728280d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c5a13411dadf4739ba09a5da11f16926" is_pre_election: true
I20260806 18:37:32.411506 4818 raft_consensus.cc:2454] T 41b7a086ba4043ecb933bd4721a649ba P c5a13411dadf4739ba09a5da11f16926 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4d2a68a0b77a43619f6206aee728280d in term 0.
I20260806 18:37:32.411507 4532 raft_consensus.cc:2454] T 41b7a086ba4043ecb933bd4721a649ba P 5fdb1c79f09d42078b9fadc4624e6d0d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4d2a68a0b77a43619f6206aee728280d in term 0.
I20260806 18:37:32.411983 4611 leader_election.cc:302] T 41b7a086ba4043ecb933bd4721a649ba P 4d2a68a0b77a43619f6206aee728280d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4d2a68a0b77a43619f6206aee728280d, 5fdb1c79f09d42078b9fadc4624e6d0d; no voters:
I20260806 18:37:32.412220 4888 raft_consensus.cc:2781] T 41b7a086ba4043ecb933bd4721a649ba P 4d2a68a0b77a43619f6206aee728280d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 18:37:32.412302 4888 raft_consensus.cc:482] T 41b7a086ba4043ecb933bd4721a649ba P 4d2a68a0b77a43619f6206aee728280d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 18:37:32.412345 4888 raft_consensus.cc:3037] T 41b7a086ba4043ecb933bd4721a649ba P 4d2a68a0b77a43619f6206aee728280d [term 0 FOLLOWER]: Advancing to term 1
I20260806 18:37:32.413362 4888 raft_consensus.cc:504] T 41b7a086ba4043ecb933bd4721a649ba P 4d2a68a0b77a43619f6206aee728280d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c5a13411dadf4739ba09a5da11f16926" member_type: VOTER last_known_addr { host: "127.19.194.3" port: 35207 } } peers { permanent_uuid: "4d2a68a0b77a43619f6206aee728280d" member_type: VOTER last_known_addr { host: "127.19.194.2" port: 46303 } } peers { permanent_uuid: "5fdb1c79f09d42078b9fadc4624e6d0d" member_type: VOTER last_known_addr { host: "127.19.194.1" port: 36577 } }
I20260806 18:37:32.413666 4888 leader_election.cc:288] T 41b7a086ba4043ecb933bd4721a649ba P 4d2a68a0b77a43619f6206aee728280d [CANDIDATE]: Term 1 election: Requested vote from peers c5a13411dadf4739ba09a5da11f16926 (127.19.194.3:35207), 5fdb1c79f09d42078b9fadc4624e6d0d (127.19.194.1:36577)
I20260806 18:37:32.413991 4818 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "41b7a086ba4043ecb933bd4721a649ba" candidate_uuid: "4d2a68a0b77a43619f6206aee728280d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c5a13411dadf4739ba09a5da11f16926"
I20260806 18:37:32.414034 4532 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "41b7a086ba4043ecb933bd4721a649ba" candidate_uuid: "4d2a68a0b77a43619f6206aee728280d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5fdb1c79f09d42078b9fadc4624e6d0d"
I20260806 18:37:32.414124 4532 raft_consensus.cc:3037] T 41b7a086ba4043ecb933bd4721a649ba P 5fdb1c79f09d42078b9fadc4624e6d0d [term 0 FOLLOWER]: Advancing to term 1
I20260806 18:37:32.414119 4818 raft_consensus.cc:3037] T 41b7a086ba4043ecb933bd4721a649ba P c5a13411dadf4739ba09a5da11f16926 [term 0 FOLLOWER]: Advancing to term 1
I20260806 18:37:32.415025 4532 raft_consensus.cc:2454] T 41b7a086ba4043ecb933bd4721a649ba P 5fdb1c79f09d42078b9fadc4624e6d0d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4d2a68a0b77a43619f6206aee728280d in term 1.
I20260806 18:37:32.415251 4818 raft_consensus.cc:2454] T 41b7a086ba4043ecb933bd4721a649ba P c5a13411dadf4739ba09a5da11f16926 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4d2a68a0b77a43619f6206aee728280d in term 1.
I20260806 18:37:32.415316 4611 leader_election.cc:302] T 41b7a086ba4043ecb933bd4721a649ba P 4d2a68a0b77a43619f6206aee728280d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4d2a68a0b77a43619f6206aee728280d, 5fdb1c79f09d42078b9fadc4624e6d0d; no voters:
I20260806 18:37:32.415512 4888 raft_consensus.cc:2781] T 41b7a086ba4043ecb933bd4721a649ba P 4d2a68a0b77a43619f6206aee728280d [term 1 FOLLOWER]: Leader election won for term 1
I20260806 18:37:32.415730 4888 raft_consensus.cc:686] T 41b7a086ba4043ecb933bd4721a649ba P 4d2a68a0b77a43619f6206aee728280d [term 1 LEADER]: Becoming Leader. State: Replica: 4d2a68a0b77a43619f6206aee728280d, State: Running, Role: LEADER
I20260806 18:37:32.416210 4888 consensus_queue.cc:237] T 41b7a086ba4043ecb933bd4721a649ba P 4d2a68a0b77a43619f6206aee728280d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c5a13411dadf4739ba09a5da11f16926" member_type: VOTER last_known_addr { host: "127.19.194.3" port: 35207 } } peers { permanent_uuid: "4d2a68a0b77a43619f6206aee728280d" member_type: VOTER last_known_addr { host: "127.19.194.2" port: 46303 } } peers { permanent_uuid: "5fdb1c79f09d42078b9fadc4624e6d0d" member_type: VOTER last_known_addr { host: "127.19.194.1" port: 36577 } }
I20260806 18:37:32.419304 4378 catalog_manager.cc:5495] T 41b7a086ba4043ecb933bd4721a649ba P 4d2a68a0b77a43619f6206aee728280d reported cstate change: term changed from 0 to 1, leader changed from <none> to 4d2a68a0b77a43619f6206aee728280d (127.19.194.2). New cstate: current_term: 1 leader_uuid: "4d2a68a0b77a43619f6206aee728280d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c5a13411dadf4739ba09a5da11f16926" member_type: VOTER last_known_addr { host: "127.19.194.3" port: 35207 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4d2a68a0b77a43619f6206aee728280d" member_type: VOTER last_known_addr { host: "127.19.194.2" port: 46303 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5fdb1c79f09d42078b9fadc4624e6d0d" member_type: VOTER last_known_addr { host: "127.19.194.1" port: 36577 } health_report { overall_health: UNKNOWN } } }
I20260806 18:37:32.464361 4880 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:37:32.461033 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41612 (local address 127.19.194.2:46303)
0806 18:37:32.461164 (+ 131us) server_negotiation.cc:206] Beginning negotiation
0806 18:37:32.461168 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 18:37:32.461182 (+ 14us) server_negotiation.cc:407] Connection header received
0806 18:37:32.461322 (+ 140us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:37:32.461325 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:37:32.461383 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:37:32.461482 (+ 99us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 18:37:32.461714 (+ 232us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:37:32.462195 (+ 481us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:37:32.463037 (+ 842us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:37:32.463252 (+ 215us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:37:32.463324 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:37:32.463610 (+ 286us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 18:37:32.464039 (+ 429us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 18:37:32.464130 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0806 18:37:32.464205 (+ 75us) server_negotiation.cc:299] Negotiation successful
0806 18:37:32.464252 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
W20260806 18:37:32.469043 4869 tablet.cc:2369] T 41b7a086ba4043ecb933bd4721a649ba P c5a13411dadf4739ba09a5da11f16926: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 18:37:32.470314 4532 raft_consensus.cc:1261] T 41b7a086ba4043ecb933bd4721a649ba P 5fdb1c79f09d42078b9fadc4624e6d0d [term 1 FOLLOWER]: Refusing update from remote peer 4d2a68a0b77a43619f6206aee728280d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 18:37:32.470331 4818 raft_consensus.cc:1261] T 41b7a086ba4043ecb933bd4721a649ba P c5a13411dadf4739ba09a5da11f16926 [term 1 FOLLOWER]: Refusing update from remote peer 4d2a68a0b77a43619f6206aee728280d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 18:37:32.470969 4888 consensus_queue.cc:1035] T 41b7a086ba4043ecb933bd4721a649ba P 4d2a68a0b77a43619f6206aee728280d [LEADER]: Connected to new peer: Peer: permanent_uuid: "5fdb1c79f09d42078b9fadc4624e6d0d" member_type: VOTER last_known_addr { host: "127.19.194.1" port: 36577 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 18:37:32.471197 4892 consensus_queue.cc:1035] T 41b7a086ba4043ecb933bd4721a649ba P 4d2a68a0b77a43619f6206aee728280d [LEADER]: Connected to new peer: Peer: permanent_uuid: "c5a13411dadf4739ba09a5da11f16926" member_type: VOTER last_known_addr { host: "127.19.194.3" port: 35207 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 18:37:32.477228 4894 mvcc.cc:205] Tried to move back new op lower bound from 7315625789311025152 to 7315625789099335680. Current Snapshot: MvccSnapshot[applied={T|T < 7315625789311025152}]
I20260806 18:37:32.477310 4895 mvcc.cc:205] Tried to move back new op lower bound from 7315625789311025152 to 7315625789099335680. Current Snapshot: MvccSnapshot[applied={T|T < 7315625789311025152}]
I20260806 18:37:32.478089 4896 mvcc.cc:205] Tried to move back new op lower bound from 7315625789311025152 to 7315625789099335680. Current Snapshot: MvccSnapshot[applied={T|T < 7315625789311025152}]
I20260806 18:37:32.487494 4378 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:41810:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 18:37:32.487650 4378 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:41810:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 18:37:32.489913 4378 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 34dfff54533849c68cf3a4743721aeb5: Sending DeleteTablet for 3 replicas of tablet 41b7a086ba4043ecb933bd4721a649ba
I20260806 18:37:32.490586 4512 tablet_service.cc:1514] Processing DeleteTablet for tablet 41b7a086ba4043ecb933bd4721a649ba with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 18:37:32 UTC) from {username='kudu', principal='kudu/127.19.194.62@KRBTEST.COM'} at 127.0.0.1:40424
I20260806 18:37:32.490691 4796 tablet_service.cc:1514] Processing DeleteTablet for tablet 41b7a086ba4043ecb933bd4721a649ba with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 18:37:32 UTC) from {username='kudu', principal='kudu/127.19.194.62@KRBTEST.COM'} at 127.0.0.1:45974
I20260806 18:37:32.490845 20232 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJW69tn/build/debug/bin/kudu with pid 4422
I20260806 18:37:32.495540 4905 tablet_replica.cc:330] T 41b7a086ba4043ecb933bd4721a649ba P c5a13411dadf4739ba09a5da11f16926: stopping tablet replica
I20260806 18:37:32.495492 4657 tablet_service.cc:1514] Processing DeleteTablet for tablet 41b7a086ba4043ecb933bd4721a649ba with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 18:37:32 UTC) from {username='kudu', principal='kudu/127.19.194.62@KRBTEST.COM'} at 127.0.0.1:41602
I20260806 18:37:32.495867 4905 raft_consensus.cc:2229] T 41b7a086ba4043ecb933bd4721a649ba P c5a13411dadf4739ba09a5da11f16926 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 18:37:32.496025 4906 tablet_replica.cc:330] T 41b7a086ba4043ecb933bd4721a649ba P 4d2a68a0b77a43619f6206aee728280d: stopping tablet replica
I20260806 18:37:32.496199 4905 raft_consensus.cc:2258] T 41b7a086ba4043ecb933bd4721a649ba P c5a13411dadf4739ba09a5da11f16926 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 18:37:32.496369 4906 raft_consensus.cc:2229] T 41b7a086ba4043ecb933bd4721a649ba P 4d2a68a0b77a43619f6206aee728280d [term 1 LEADER]: Raft consensus shutting down.
I20260806 18:37:32.496829 4906 raft_consensus.cc:2258] T 41b7a086ba4043ecb933bd4721a649ba P 4d2a68a0b77a43619f6206aee728280d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 18:37:32.497681 4905 ts_tablet_manager.cc:1905] T 41b7a086ba4043ecb933bd4721a649ba P c5a13411dadf4739ba09a5da11f16926: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 18:37:32.498287 4906 ts_tablet_manager.cc:1905] T 41b7a086ba4043ecb933bd4721a649ba P 4d2a68a0b77a43619f6206aee728280d: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260806 18:37:32.499787 4362 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.19.194.1:36577 (error 108)
I20260806 18:37:32.500077 20232 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJW69tn/build/debug/bin/kudu with pid 4589
I20260806 18:37:32.502574 4877 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:37:32.502300 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.194.1:36577 (local address 127.0.0.1:40438)
0806 18:37:32.502435 (+ 135us) negotiation.cc:107] Waiting for socket to connect
0806 18:37:32.502500 (+ 65us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.194.1:36577: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":53}
I20260806 18:37:32.503885 4905 ts_tablet_manager.cc:1918] T 41b7a086ba4043ecb933bd4721a649ba P c5a13411dadf4739ba09a5da11f16926: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 18:37:32.504019 4905 log.cc:1196] T 41b7a086ba4043ecb933bd4721a649ba P c5a13411dadf4739ba09a5da11f16926: Deleting WAL directory at /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786041246094603-20232-0/minicluster-data/ts-2/wal/wals/41b7a086ba4043ecb933bd4721a649ba
I20260806 18:37:32.504383 4905 ts_tablet_manager.cc:1939] T 41b7a086ba4043ecb933bd4721a649ba P c5a13411dadf4739ba09a5da11f16926: Deleting consensus metadata
I20260806 18:37:32.505594 4363 catalog_manager.cc:4841] TS c5a13411dadf4739ba09a5da11f16926 (127.19.194.3:35207): tablet 41b7a086ba4043ecb933bd4721a649ba (table test-table [id=73c0eab94a454780b9b697943bae4e77]) successfully deleted
W20260806 18:37:32.505856 4362 catalog_manager.cc:4584] TS 5fdb1c79f09d42078b9fadc4624e6d0d (127.19.194.1:36577): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 41b7a086ba4043ecb933bd4721a649ba: Network error: Client connection negotiation failed: client connection to 127.19.194.1:36577: connect: Connection refused (error 111)
I20260806 18:37:32.508370 20232 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJW69tn/build/debug/bin/kudu with pid 4731
I20260806 18:37:32.508581 4877 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:37:32.508307 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.194.2:46303 (local address 127.0.0.1:41620)
0806 18:37:32.508420 (+ 113us) negotiation.cc:107] Waiting for socket to connect
0806 18:37:32.508498 (+ 78us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.194.2:46303: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":45}
I20260806 18:37:32.514892 20232 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJW69tn/build/debug/bin/kudu with pid 4343
I20260806 18:37:52.522179 20232 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJW69tn/build/debug/bin/kudu
/tmp/dist-test-taskJW69tn/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786041246094603-20232-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786041246094603-20232-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786041246094603-20232-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786041246094603-20232-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.19.194.62:36421
--webserver_interface=127.19.194.62
--webserver_port=39513
--keytab_file=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786041246094603-20232-0/krb5kdc/kudu_127.19.194.62.keytab
--principal=kudu/127.19.194.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.19.194.20:36777
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.19.194.62:36421
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20 with env {KRB5CCNAME=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786041246094603-20232-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786041246094603-20232-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786041246094603-20232-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 18:37:52.632455 4908 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 18:37:52.632714 4908 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 18:37:52.632753 4908 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 18:37:52.636648 4908 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260806 18:37:52.636694 4908 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260806 18:37:52.636710 4908 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260806 18:37:52.636723 4908 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 18:37:52.636741 4908 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 18:37:52.636760 4908 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 18:37:52.636781 4908 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 18:37:52.636801 4908 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 18:37:52.641862 4908 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.194.20:36777
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786041246094603-20232-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786041246094603-20232-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.19.194.62:36421
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786041246094603-20232-0/krb5kdc/kudu_127.19.194.62.keytab
--principal=kudu/127.19.194.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.19.194.62:36421
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786041246094603-20232-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.19.194.62
--webserver_port=39513
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4908
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786041246094603-20232-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 17:59:13 UTC on ccycloud.quasar-wvsnzx.root.comops.site
build id 4571028
I20260806 18:37:52.642784 4908 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 18:37:52.643807 4908 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 18:37:52.652237 4916 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:37:52.652264 4914 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 18:37:52.652482 4908 server_base.cc:1034] running on GCE node
W20260806 18:37:52.652426 4913 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 18:37:52.653096 4908 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 18:37:52.654263 4908 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 18:37:52.655452 4908 hybrid_clock.cc:648] HybridClock initialized: now 1786041472655420 us; error 45 us; skew 500 ppm
Aug 06 18:37:52 dist-test-slave-1xrh krb5kdc[3760](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041472, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.194.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 18:37:52.658161 4908 init.cc:377] Logged in from keytab as kudu/127.19.194.62@KRBTEST.COM (short username kudu)
I20260806 18:37:52.659034 4908 webserver.cc:460] Webserver started at http://127.19.194.62:39513/ using document root <none> and password file <none>
I20260806 18:37:52.659615 4908 fs_manager.cc:359] Metadata directory not provided
I20260806 18:37:52.659711 4908 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 18:37:52.663364 4908 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.004s sys 0.000s
I20260806 18:37:52.665369 4923 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 18:37:52.666568 4908 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260806 18:37:52.666706 4908 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786041246094603-20232-0/minicluster-data/master-0/data,/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786041246094603-20232-0/minicluster-data/master-0/wal
uuid: "34dfff54533849c68cf3a4743721aeb5"
format_stamp: "Formatted at 2026-08-06 18:37:10 on dist-test-slave-1xrh"
server_key: "6ac8ff971ade94db00b2ef63baf3c45d"
server_key_iv: "c0bad579135a7450618b2f3d295e550d"
server_key_version: "encryptionkey@0"
I20260806 18:37:52.667284 4908 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786041246094603-20232-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786041246094603-20232-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786041246094603-20232-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 18:37:52.680827 4908 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 18:37:52.683966 4908 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 18:37:52.684165 4908 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 18:37:52.691149 4908 rpc_server.cc:307] RPC server started. Bound to: 127.19.194.62:36421
I20260806 18:37:52.691463 4978 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.194.62:36421 every 8 connection(s)
I20260806 18:37:52.692178 4908 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786041246094603-20232-0/minicluster-data/master-0/data/info.pb
I20260806 18:37:52.696265 4979 sys_catalog.cc:263] Verifying existing consensus state
I20260806 18:37:52.697806 4979 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 34dfff54533849c68cf3a4743721aeb5: Bootstrap starting.
I20260806 18:37:52.699556 20232 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJW69tn/build/debug/bin/kudu as pid 4908
I20260806 18:37:52.699975 20232 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJW69tn/build/debug/bin/kudu
/tmp/dist-test-taskJW69tn/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786041246094603-20232-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786041246094603-20232-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786041246094603-20232-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786041246094603-20232-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.19.194.1:36577
--local_ip_for_outbound_sockets=127.19.194.1
--tserver_master_addrs=127.19.194.62:36421
--webserver_port=40163
--webserver_interface=127.19.194.1
--keytab_file=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786041246094603-20232-0/krb5kdc/kudu_127.19.194.1.keytab
--principal=kudu/127.19.194.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.19.194.20:36777
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786041246094603-20232-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786041246094603-20232-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786041246094603-20232-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260806 18:37:52.715246 4979 log.cc:824] T 00000000000000000000000000000000 P 34dfff54533849c68cf3a4743721aeb5: Log is configured to *not* fsync() on all Append() calls
I20260806 18:37:52.725980 4979 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 34dfff54533849c68cf3a4743721aeb5: Bootstrap replayed 1/1 log segments. Stats: ops{read=14 overwritten=0 applied=14 ignored=0} inserts{seen=8 ignored=0} mutations{seen=8 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260806 18:37:52.726341 4979 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 34dfff54533849c68cf3a4743721aeb5: Bootstrap complete.
I20260806 18:37:52.730319 4979 raft_consensus.cc:348] T 00000000000000000000000000000000 P 34dfff54533849c68cf3a4743721aeb5 [term 2 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "34dfff54533849c68cf3a4743721aeb5" member_type: VOTER last_known_addr { host: "127.19.194.62" port: 36421 } }
I20260806 18:37:52.730711 4979 raft_consensus.cc:729] T 00000000000000000000000000000000 P 34dfff54533849c68cf3a4743721aeb5 [term 2 FOLLOWER]: Becoming Follower/Learner. State: Replica: 34dfff54533849c68cf3a4743721aeb5, State: Initialized, Role: FOLLOWER
I20260806 18:37:52.731271 4979 consensus_queue.cc:260] T 00000000000000000000000000000000 P 34dfff54533849c68cf3a4743721aeb5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "34dfff54533849c68cf3a4743721aeb5" member_type: VOTER last_known_addr { host: "127.19.194.62" port: 36421 } }
I20260806 18:37:52.731417 4979 raft_consensus.cc:388] T 00000000000000000000000000000000 P 34dfff54533849c68cf3a4743721aeb5 [term 2 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 18:37:52.731560 4979 raft_consensus.cc:482] T 00000000000000000000000000000000 P 34dfff54533849c68cf3a4743721aeb5 [term 2 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 18:37:52.731683 4979 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 34dfff54533849c68cf3a4743721aeb5 [term 2 FOLLOWER]: Advancing to term 3
I20260806 18:37:52.733614 4979 raft_consensus.cc:504] T 00000000000000000000000000000000 P 34dfff54533849c68cf3a4743721aeb5 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "34dfff54533849c68cf3a4743721aeb5" member_type: VOTER last_known_addr { host: "127.19.194.62" port: 36421 } }
I20260806 18:37:52.733943 4979 leader_election.cc:302] T 00000000000000000000000000000000 P 34dfff54533849c68cf3a4743721aeb5 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 34dfff54533849c68cf3a4743721aeb5; no voters:
I20260806 18:37:52.734273 4979 leader_election.cc:288] T 00000000000000000000000000000000 P 34dfff54533849c68cf3a4743721aeb5 [CANDIDATE]: Term 3 election: Requested vote from peers
I20260806 18:37:52.734364 4983 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 34dfff54533849c68cf3a4743721aeb5 [term 3 FOLLOWER]: Leader election won for term 3
I20260806 18:37:52.734606 4983 raft_consensus.cc:686] T 00000000000000000000000000000000 P 34dfff54533849c68cf3a4743721aeb5 [term 3 LEADER]: Becoming Leader. State: Replica: 34dfff54533849c68cf3a4743721aeb5, State: Running, Role: LEADER
I20260806 18:37:52.734889 4983 consensus_queue.cc:237] T 00000000000000000000000000000000 P 34dfff54533849c68cf3a4743721aeb5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 3, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "34dfff54533849c68cf3a4743721aeb5" member_type: VOTER last_known_addr { host: "127.19.194.62" port: 36421 } }
I20260806 18:37:52.735491 4979 sys_catalog.cc:564] T 00000000000000000000000000000000 P 34dfff54533849c68cf3a4743721aeb5 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 18:37:52.735781 4983 sys_catalog.cc:455] T 00000000000000000000000000000000 P 34dfff54533849c68cf3a4743721aeb5 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 34dfff54533849c68cf3a4743721aeb5. Latest consensus state: current_term: 3 leader_uuid: "34dfff54533849c68cf3a4743721aeb5" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "34dfff54533849c68cf3a4743721aeb5" member_type: VOTER last_known_addr { host: "127.19.194.62" port: 36421 } } }
I20260806 18:37:52.735905 4983 sys_catalog.cc:458] T 00000000000000000000000000000000 P 34dfff54533849c68cf3a4743721aeb5 [sys.catalog]: This master's current role is: LEADER
I20260806 18:37:52.736584 4986 sys_catalog.cc:455] T 00000000000000000000000000000000 P 34dfff54533849c68cf3a4743721aeb5 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 3 leader_uuid: "34dfff54533849c68cf3a4743721aeb5" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "34dfff54533849c68cf3a4743721aeb5" member_type: VOTER last_known_addr { host: "127.19.194.62" port: 36421 } } }
I20260806 18:37:52.736680 4986 sys_catalog.cc:458] T 00000000000000000000000000000000 P 34dfff54533849c68cf3a4743721aeb5 [sys.catalog]: This master's current role is: LEADER
I20260806 18:37:52.737277 4989 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 18:37:52.742203 4989 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 18:37:52.743149 4989 catalog_manager.cc:1259] Loaded cluster ID: 9dc90a31f80745688367d0fd1a89a281
I20260806 18:37:52.743211 4989 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 18:37:52.745906 4989 catalog_manager.cc:1504] Loading token signing keys...
I20260806 18:37:52.746682 4989 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 34dfff54533849c68cf3a4743721aeb5: Loaded TSK: 0
I20260806 18:37:52.746768 4989 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 34dfff54533849c68cf3a4743721aeb5: Loaded TSK: 1
F20260806 18:37:52.747491 4989 token_signing_key.cc:66] Check failed: _s.ok() Bad status: Runtime error: unable to load data from memory: error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=RSA error:04093004:rsa routines:OLD_RSA_PRIV_DECODE:RSA lib:rsa_ameth.c:121 error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=PKCS8_PRIV_KEY_INFO
*** Check failure stack trace: ***
@ 0x7f6a72d409ad google::LogMessage::Fail() at ??:0
@ 0x7f6a72d4388a google::LogMessage::SendToLog() at ??:0
@ 0x7f6a72d404de google::LogMessage::Flush() at ??:0
@ 0x7f6a72d41a19 google::LogMessageFatal::~LogMessageFatal() at ??:0
@ 0x7f6a75a0418d kudu::security::TokenSigningPrivateKey::TokenSigningPrivateKey() at ??:0
@ 0x7f6a759fcf0d kudu::security::TokenSigner::ImportKeys() at ??:0
@ 0x7f6a7d6a57bb kudu::master::CatalogManager::LoadTskEntries() at ??:0
@ 0x7f6a7d67f99c kudu::master::CatalogManager::InitTokenSigner() at ??:0
@ 0x7f6a7d680178 _ZZN4kudu6master14CatalogManager24PrepareForLeadershipTaskEvENKUlvE3_clEv at ??:0
@ 0x7f6a7d6c1df8 _ZNSt17_Function_handlerIFN4kudu6StatusEvEZNS0_6master14CatalogManager24PrepareForLeadershipTaskEvEUlvE3_E9_M_invokeERKSt9_Any_data at ??:0
@ 0x7f6a7d6e082b std::function<>::operator()() at ??:0
@ 0x7f6a7d67fb4f _ZZN4kudu6master14CatalogManager24PrepareForLeadershipTaskEvENKUlSt8functionIFNS_6StatusEvEERKNS_9consensus13RaftConsensusElPKcE_clES5_S9_lSB_ at ??:0
@ 0x7f6a7d680f3e kudu::master::CatalogManager::PrepareForLeadershipTask() at ??:0
@ 0x7f6a7d67d579 _ZZN4kudu6master14CatalogManager17ElectedAsLeaderCbEvENKUlvE_clEv at ??:0
@ 0x7f6a7d6c1640 _ZNSt17_Function_handlerIFvvEZN4kudu6master14CatalogManager17ElectedAsLeaderCbEvEUlvE_E9_M_invokeERKSt9_Any_data at ??:0
@ 0x61d838 std::function<>::operator()() at ??:0
@ 0x7f6a74f923d7 kudu::ThreadPool::DispatchThread() at ??:0
@ 0x7f6a74f92c83 _ZZN4kudu10ThreadPool12CreateThreadEvENKUlvE_clEv at ??:0
@ 0x7f6a74f943f2 _ZNSt17_Function_handlerIFvvEZN4kudu10ThreadPool12CreateThreadEvEUlvE_E9_M_invokeERKSt9_Any_data at ??:0
@ 0x61d838 std::function<>::operator()() at ??:0
@ 0x7f6a74f83512 kudu::Thread::SuperviseThread() at ??:0
@ 0x7f6a765c76db start_thread at ??:0
@ 0x7f6a7183e61f clone at ??:0
W20260806 18:37:52.849761 4981 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 18:37:52.850000 4981 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 18:37:52.850042 4981 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 18:37:52.854218 4981 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 18:37:52.854270 4981 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 18:37:52.854369 4981 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.19.194.1
I20260806 18:37:52.859653 4981 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.194.20:36777
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786041246094603-20232-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786041246094603-20232-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786041246094603-20232-0/krb5kdc/kudu_127.19.194.1.keytab
--principal=kudu/127.19.194.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.19.194.1:36577
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786041246094603-20232-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.19.194.1
--webserver_port=40163
--tserver_master_addrs=127.19.194.62:36421
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4981
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.19.194.1
--log_dir=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786041246094603-20232-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 17:59:13 UTC on ccycloud.quasar-wvsnzx.root.comops.site
build id 4571028
I20260806 18:37:52.860654 4981 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 18:37:52.861678 4981 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 18:37:52.872259 5030 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:37:52.872387 5033 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:37:52.872269 5031 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 18:37:52.872670 4981 server_base.cc:1034] running on GCE node
I20260806 18:37:52.873055 4981 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 18:37:52.873692 4981 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 18:37:52.874876 4981 hybrid_clock.cc:648] HybridClock initialized: now 1786041472874855 us; error 35 us; skew 500 ppm
Aug 06 18:37:52 dist-test-slave-1xrh krb5kdc[3760](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041472, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.194.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 18:37:52.877643 4981 init.cc:377] Logged in from keytab as kudu/127.19.194.1@KRBTEST.COM (short username kudu)
I20260806 18:37:52.878489 4981 webserver.cc:460] Webserver started at http://127.19.194.1:40163/ using document root <none> and password file <none>
I20260806 18:37:52.879029 4981 fs_manager.cc:359] Metadata directory not provided
I20260806 18:37:52.879089 4981 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 18:37:52.882825 4981 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.003s sys 0.000s
I20260806 18:37:52.884972 5040 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 18:37:52.886193 4981 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260806 18:37:52.886282 4981 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786041246094603-20232-0/minicluster-data/ts-0/data,/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786041246094603-20232-0/minicluster-data/ts-0/wal
uuid: "5fdb1c79f09d42078b9fadc4624e6d0d"
format_stamp: "Formatted at 2026-08-06 18:37:10 on dist-test-slave-1xrh"
server_key: "18754be74733259eab711d3b24a590eb"
server_key_iv: "532b38fc980d1832027c7f020488d752"
server_key_version: "encryptionkey@0"
I20260806 18:37:52.886865 4981 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786041246094603-20232-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786041246094603-20232-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786041246094603-20232-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 18:37:52.915691 4981 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 18:37:52.918757 4981 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 18:37:52.918967 4981 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 18:37:52.919742 4981 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 18:37:52.921866 5047 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260806 18:37:52.927178 4981 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260806 18:37:52.927243 4981 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.006s user 0.001s sys 0.000s
I20260806 18:37:52.927320 4981 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260806 18:37:52.928946 4981 ts_tablet_manager.cc:610] Registered 1 tablets
I20260806 18:37:52.929015 4981 ts_tablet_manager.cc:589] Time spent register tablets: real 0.002s user 0.000s sys 0.000s
I20260806 18:37:52.929306 5047 tablet_bootstrap.cc:492] T 41b7a086ba4043ecb933bd4721a649ba P 5fdb1c79f09d42078b9fadc4624e6d0d: Bootstrap starting.
I20260806 18:37:52.940143 4981 rpc_server.cc:307] RPC server started. Bound to: 127.19.194.1:36577
I20260806 18:37:52.940220 5166 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.194.1:36577 every 8 connection(s)
I20260806 18:37:52.941097 4981 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786041246094603-20232-0/minicluster-data/ts-0/data/info.pb
I20260806 18:37:52.947391 20232 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJW69tn/build/debug/bin/kudu as pid 4981
I20260806 18:37:52.947799 20232 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJW69tn/build/debug/bin/kudu
/tmp/dist-test-taskJW69tn/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786041246094603-20232-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786041246094603-20232-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786041246094603-20232-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786041246094603-20232-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.19.194.2:46303
--local_ip_for_outbound_sockets=127.19.194.2
--tserver_master_addrs=127.19.194.62:36421
--webserver_port=38271
--webserver_interface=127.19.194.2
--keytab_file=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786041246094603-20232-0/krb5kdc/kudu_127.19.194.2.keytab
--principal=kudu/127.19.194.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.19.194.20:36777
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786041246094603-20232-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786041246094603-20232-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786041246094603-20232-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260806 18:37:52.953040 5047 log.cc:824] T 41b7a086ba4043ecb933bd4721a649ba P 5fdb1c79f09d42078b9fadc4624e6d0d: Log is configured to *not* fsync() on all Append() calls
I20260806 18:37:52.960031 5047 tablet_bootstrap.cc:492] T 41b7a086ba4043ecb933bd4721a649ba P 5fdb1c79f09d42078b9fadc4624e6d0d: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260806 18:37:52.960390 5047 tablet_bootstrap.cc:492] T 41b7a086ba4043ecb933bd4721a649ba P 5fdb1c79f09d42078b9fadc4624e6d0d: Bootstrap complete.
I20260806 18:37:52.961542 5047 ts_tablet_manager.cc:1397] T 41b7a086ba4043ecb933bd4721a649ba P 5fdb1c79f09d42078b9fadc4624e6d0d: Time spent bootstrapping tablet: real 0.032s user 0.020s sys 0.008s
I20260806 18:37:52.963984 5047 raft_consensus.cc:348] T 41b7a086ba4043ecb933bd4721a649ba P 5fdb1c79f09d42078b9fadc4624e6d0d [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c5a13411dadf4739ba09a5da11f16926" member_type: VOTER last_known_addr { host: "127.19.194.3" port: 35207 } } peers { permanent_uuid: "4d2a68a0b77a43619f6206aee728280d" member_type: VOTER last_known_addr { host: "127.19.194.2" port: 46303 } } peers { permanent_uuid: "5fdb1c79f09d42078b9fadc4624e6d0d" member_type: VOTER last_known_addr { host: "127.19.194.1" port: 36577 } }
I20260806 18:37:52.964391 5047 raft_consensus.cc:729] T 41b7a086ba4043ecb933bd4721a649ba P 5fdb1c79f09d42078b9fadc4624e6d0d [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5fdb1c79f09d42078b9fadc4624e6d0d, State: Initialized, Role: FOLLOWER
I20260806 18:37:52.964854 5047 consensus_queue.cc:260] T 41b7a086ba4043ecb933bd4721a649ba P 5fdb1c79f09d42078b9fadc4624e6d0d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c5a13411dadf4739ba09a5da11f16926" member_type: VOTER last_known_addr { host: "127.19.194.3" port: 35207 } } peers { permanent_uuid: "4d2a68a0b77a43619f6206aee728280d" member_type: VOTER last_known_addr { host: "127.19.194.2" port: 46303 } } peers { permanent_uuid: "5fdb1c79f09d42078b9fadc4624e6d0d" member_type: VOTER last_known_addr { host: "127.19.194.1" port: 36577 } }
I20260806 18:37:52.965659 5047 ts_tablet_manager.cc:1428] T 41b7a086ba4043ecb933bd4721a649ba P 5fdb1c79f09d42078b9fadc4624e6d0d: Time spent starting tablet: real 0.004s user 0.006s sys 0.000s
W20260806 18:37:52.986788 5169 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 18:37:52.943296 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.194.62:36421 (local address 127.19.194.1:33327)
0806 18:37:52.943851 (+ 555us) negotiation.cc:107] Waiting for socket to connect
0806 18:37:52.943905 (+ 54us) client_negotiation.cc:174] Beginning negotiation
0806 18:37:52.944706 (+ 801us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:37:52.985843 (+ 41137us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.194.62:36421: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
Metrics: {"client-negotiator.queue_time_us":280,"thread_start_us":142,"threads_started":1}
W20260806 18:37:52.987447 5167 heartbeater.cc:644] Failed to heartbeat to 127.19.194.62:36421 (0 consecutive failures): Network error: Failed to ping master at 127.19.194.62:36421: Client connection negotiation failed: client connection to 127.19.194.62:36421: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260806 18:37:52.988204 5169 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:37:52.987920 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.194.62:36421 (local address 127.19.194.1:60527)
0806 18:37:52.988064 (+ 144us) negotiation.cc:107] Waiting for socket to connect
0806 18:37:52.988139 (+ 75us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.194.62:36421: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":54}
I20260806 18:37:52.988903 5169 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:37:52.988648 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.194.62:36421 (local address 127.19.194.1:53645)
0806 18:37:52.988768 (+ 120us) negotiation.cc:107] Waiting for socket to connect
0806 18:37:52.988830 (+ 62us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.194.62:36421: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":50}
W20260806 18:37:52.989120 5167 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260806 18:37:53.067780 5170 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 18:37:53.067988 5170 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 18:37:53.068027 5170 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 18:37:53.071835 5170 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 18:37:53.071879 5170 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 18:37:53.071933 5170 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.19.194.2
I20260806 18:37:53.076722 5170 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.194.20:36777
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786041246094603-20232-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786041246094603-20232-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786041246094603-20232-0/krb5kdc/kudu_127.19.194.2.keytab
--principal=kudu/127.19.194.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.19.194.2:46303
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786041246094603-20232-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.19.194.2
--webserver_port=38271
--tserver_master_addrs=127.19.194.62:36421
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5170
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.19.194.2
--log_dir=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786041246094603-20232-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 17:59:13 UTC on ccycloud.quasar-wvsnzx.root.comops.site
build id 4571028
I20260806 18:37:53.077545 5170 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 18:37:53.078387 5170 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 18:37:53.087513 5181 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 18:37:53.087707 5170 server_base.cc:1034] running on GCE node
W20260806 18:37:53.087534 5179 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:37:53.087879 5178 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 18:37:53.088280 5170 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 18:37:53.088852 5170 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 18:37:53.090046 5170 hybrid_clock.cc:648] HybridClock initialized: now 1786041473090028 us; error 29 us; skew 500 ppm
Aug 06 18:37:53 dist-test-slave-1xrh krb5kdc[3760](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041473, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.194.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 18:37:53.092729 5170 init.cc:377] Logged in from keytab as kudu/127.19.194.2@KRBTEST.COM (short username kudu)
I20260806 18:37:53.093501 5170 webserver.cc:460] Webserver started at http://127.19.194.2:38271/ using document root <none> and password file <none>
I20260806 18:37:53.094012 5170 fs_manager.cc:359] Metadata directory not provided
I20260806 18:37:53.094100 5170 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 18:37:53.097698 5170 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.002s sys 0.000s
I20260806 18:37:53.099776 5188 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 18:37:53.100889 5170 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260806 18:37:53.101043 5170 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786041246094603-20232-0/minicluster-data/ts-1/data,/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786041246094603-20232-0/minicluster-data/ts-1/wal
uuid: "4d2a68a0b77a43619f6206aee728280d"
format_stamp: "Formatted at 2026-08-06 18:37:10 on dist-test-slave-1xrh"
server_key: "f9d523d313ee70f29dd5f5475c7ff29c"
server_key_iv: "313833d9c7638513ffd6527402988301"
server_key_version: "encryptionkey@0"
I20260806 18:37:53.101538 5170 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786041246094603-20232-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786041246094603-20232-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786041246094603-20232-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 18:37:53.115427 5170 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 18:37:53.118505 5170 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 18:37:53.118697 5170 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 18:37:53.119303 5170 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 18:37:53.120304 5195 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260806 18:37:53.124162 5170 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260806 18:37:53.124238 5170 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.004s user 0.000s sys 0.000s
I20260806 18:37:53.124333 5170 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260806 18:37:53.125861 5170 ts_tablet_manager.cc:610] Registered 1 tablets
I20260806 18:37:53.125941 5170 ts_tablet_manager.cc:589] Time spent register tablets: real 0.002s user 0.000s sys 0.000s
I20260806 18:37:53.126170 5195 tablet_bootstrap.cc:492] T 41b7a086ba4043ecb933bd4721a649ba P 4d2a68a0b77a43619f6206aee728280d: Bootstrap starting.
I20260806 18:37:53.130993 5195 log.cc:824] T 41b7a086ba4043ecb933bd4721a649ba P 4d2a68a0b77a43619f6206aee728280d: Log is configured to *not* fsync() on all Append() calls
I20260806 18:37:53.136758 5170 rpc_server.cc:307] RPC server started. Bound to: 127.19.194.2:46303
I20260806 18:37:53.136790 5308 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.194.2:46303 every 8 connection(s)
I20260806 18:37:53.137553 5170 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786041246094603-20232-0/minicluster-data/ts-1/data/info.pb
I20260806 18:37:53.139389 5195 tablet_bootstrap.cc:492] T 41b7a086ba4043ecb933bd4721a649ba P 4d2a68a0b77a43619f6206aee728280d: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260806 18:37:53.139845 5195 tablet_bootstrap.cc:492] T 41b7a086ba4043ecb933bd4721a649ba P 4d2a68a0b77a43619f6206aee728280d: Bootstrap complete.
I20260806 18:37:53.140187 5195 ts_tablet_manager.cc:1397] T 41b7a086ba4043ecb933bd4721a649ba P 4d2a68a0b77a43619f6206aee728280d: Time spent bootstrapping tablet: real 0.014s user 0.013s sys 0.000s
I20260806 18:37:53.140893 5311 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:37:53.139378 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.194.62:36421 (local address 127.19.194.2:34381)
0806 18:37:53.139925 (+ 547us) negotiation.cc:107] Waiting for socket to connect
0806 18:37:53.140059 (+ 134us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.194.62:36421: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":315,"thread_start_us":165,"threads_started":1}
W20260806 18:37:53.141400 5309 heartbeater.cc:644] Failed to heartbeat to 127.19.194.62:36421 (0 consecutive failures): Network error: Failed to ping master at 127.19.194.62:36421: Client connection negotiation failed: client connection to 127.19.194.62:36421: connect: Connection refused (error 111)
I20260806 18:37:53.142055 5311 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:37:53.141803 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.194.62:36421 (local address 127.19.194.2:51789)
0806 18:37:53.141933 (+ 130us) negotiation.cc:107] Waiting for socket to connect
0806 18:37:53.141994 (+ 61us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.194.62:36421: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":49}
I20260806 18:37:53.142668 5311 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:37:53.142435 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.194.62:36421 (local address 127.19.194.2:46245)
0806 18:37:53.142544 (+ 109us) negotiation.cc:107] Waiting for socket to connect
0806 18:37:53.142598 (+ 54us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.194.62:36421: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":44}
W20260806 18:37:53.142835 5309 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260806 18:37:53.143625 5195 raft_consensus.cc:348] T 41b7a086ba4043ecb933bd4721a649ba P 4d2a68a0b77a43619f6206aee728280d [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c5a13411dadf4739ba09a5da11f16926" member_type: VOTER last_known_addr { host: "127.19.194.3" port: 35207 } } peers { permanent_uuid: "4d2a68a0b77a43619f6206aee728280d" member_type: VOTER last_known_addr { host: "127.19.194.2" port: 46303 } } peers { permanent_uuid: "5fdb1c79f09d42078b9fadc4624e6d0d" member_type: VOTER last_known_addr { host: "127.19.194.1" port: 36577 } }
I20260806 18:37:53.144013 5195 raft_consensus.cc:729] T 41b7a086ba4043ecb933bd4721a649ba P 4d2a68a0b77a43619f6206aee728280d [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4d2a68a0b77a43619f6206aee728280d, State: Initialized, Role: FOLLOWER
I20260806 18:37:53.144369 5195 consensus_queue.cc:260] T 41b7a086ba4043ecb933bd4721a649ba P 4d2a68a0b77a43619f6206aee728280d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c5a13411dadf4739ba09a5da11f16926" member_type: VOTER last_known_addr { host: "127.19.194.3" port: 35207 } } peers { permanent_uuid: "4d2a68a0b77a43619f6206aee728280d" member_type: VOTER last_known_addr { host: "127.19.194.2" port: 46303 } } peers { permanent_uuid: "5fdb1c79f09d42078b9fadc4624e6d0d" member_type: VOTER last_known_addr { host: "127.19.194.1" port: 36577 } }
I20260806 18:37:53.145187 5195 ts_tablet_manager.cc:1428] T 41b7a086ba4043ecb933bd4721a649ba P 4d2a68a0b77a43619f6206aee728280d: Time spent starting tablet: real 0.005s user 0.005s sys 0.000s
I20260806 18:37:53.145323 5311 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:37:53.145086 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.194.62:36421 (local address 127.19.194.2:38181)
0806 18:37:53.145218 (+ 132us) negotiation.cc:107] Waiting for socket to connect
0806 18:37:53.145272 (+ 54us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.194.62:36421: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":44}
I20260806 18:37:53.145871 5311 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:37:53.145692 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.194.62:36421 (local address 127.19.194.2:58075)
0806 18:37:53.145787 (+ 95us) negotiation.cc:107] Waiting for socket to connect
0806 18:37:53.145827 (+ 40us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.194.62:36421: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":41}
I20260806 18:37:53.145886 20232 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJW69tn/build/debug/bin/kudu as pid 5170
I20260806 18:37:53.146311 20232 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJW69tn/build/debug/bin/kudu
/tmp/dist-test-taskJW69tn/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786041246094603-20232-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786041246094603-20232-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786041246094603-20232-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786041246094603-20232-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.19.194.3:35207
--local_ip_for_outbound_sockets=127.19.194.3
--tserver_master_addrs=127.19.194.62:36421
--webserver_port=33259
--webserver_interface=127.19.194.3
--keytab_file=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786041246094603-20232-0/krb5kdc/kudu_127.19.194.3.keytab
--principal=kudu/127.19.194.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.19.194.20:36777
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786041246094603-20232-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786041246094603-20232-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786041246094603-20232-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 18:37:53.252875 5313 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 18:37:53.253084 5313 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 18:37:53.253140 5313 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 18:37:53.257102 5313 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 18:37:53.257166 5313 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 18:37:53.257248 5313 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.19.194.3
I20260806 18:37:53.262210 5313 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.194.20:36777
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786041246094603-20232-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786041246094603-20232-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786041246094603-20232-0/krb5kdc/kudu_127.19.194.3.keytab
--principal=kudu/127.19.194.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.19.194.3:35207
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786041246094603-20232-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.19.194.3
--webserver_port=33259
--tserver_master_addrs=127.19.194.62:36421
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5313
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.19.194.3
--log_dir=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786041246094603-20232-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 17:59:13 UTC on ccycloud.quasar-wvsnzx.root.comops.site
build id 4571028
I20260806 18:37:53.263159 5313 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 18:37:53.264125 5313 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 18:37:53.273859 5319 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:37:53.273876 5321 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:37:53.273864 5318 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 18:37:53.274235 5313 server_base.cc:1034] running on GCE node
I20260806 18:37:53.274602 5313 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 18:37:53.275247 5313 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 18:37:53.276422 5313 hybrid_clock.cc:648] HybridClock initialized: now 1786041473276412 us; error 28 us; skew 500 ppm
Aug 06 18:37:53 dist-test-slave-1xrh krb5kdc[3760](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041473, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.194.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 18:37:53.278965 5313 init.cc:377] Logged in from keytab as kudu/127.19.194.3@KRBTEST.COM (short username kudu)
I20260806 18:37:53.279795 5313 webserver.cc:460] Webserver started at http://127.19.194.3:33259/ using document root <none> and password file <none>
I20260806 18:37:53.280346 5313 fs_manager.cc:359] Metadata directory not provided
I20260806 18:37:53.280436 5313 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 18:37:53.284034 5313 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.001s sys 0.003s
I20260806 18:37:53.285948 5328 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 18:37:53.287039 5313 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260806 18:37:53.287211 5313 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786041246094603-20232-0/minicluster-data/ts-2/data,/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786041246094603-20232-0/minicluster-data/ts-2/wal
uuid: "c5a13411dadf4739ba09a5da11f16926"
format_stamp: "Formatted at 2026-08-06 18:37:11 on dist-test-slave-1xrh"
server_key: "702a906614a2f7717547b4cb83b505a1"
server_key_iv: "8fafdc8e52d92aa949a83dcb793bd92c"
server_key_version: "encryptionkey@0"
I20260806 18:37:53.287724 5313 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786041246094603-20232-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786041246094603-20232-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786041246094603-20232-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 18:37:53.308247 5313 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 18:37:53.311883 5313 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 18:37:53.312098 5313 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 18:37:53.312883 5313 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 18:37:53.314000 5313 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 18:37:53.314055 5313 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 18:37:53.314111 5313 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 18:37:53.314147 5313 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 18:37:53.324163 5313 rpc_server.cc:307] RPC server started. Bound to: 127.19.194.3:35207
I20260806 18:37:53.324187 5447 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.194.3:35207 every 8 connection(s)
I20260806 18:37:53.325166 5313 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786041246094603-20232-0/minicluster-data/ts-2/data/info.pb
I20260806 18:37:53.329352 5450 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:37:53.327701 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.194.62:36421 (local address 127.19.194.3:57431)
0806 18:37:53.328566 (+ 865us) negotiation.cc:107] Waiting for socket to connect
0806 18:37:53.328672 (+ 106us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.194.62:36421: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":671,"thread_start_us":530,"threads_started":1}
W20260806 18:37:53.329802 5448 heartbeater.cc:644] Failed to heartbeat to 127.19.194.62:36421 (0 consecutive failures): Network error: Failed to ping master at 127.19.194.62:36421: Client connection negotiation failed: client connection to 127.19.194.62:36421: connect: Connection refused (error 111)
I20260806 18:37:53.330387 5450 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:37:53.330187 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.194.62:36421 (local address 127.19.194.3:36415)
0806 18:37:53.330287 (+ 100us) negotiation.cc:107] Waiting for socket to connect
0806 18:37:53.330333 (+ 46us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.194.62:36421: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":38}
I20260806 18:37:53.330997 5450 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:37:53.330807 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.194.62:36421 (local address 127.19.194.3:36793)
0806 18:37:53.330900 (+ 93us) negotiation.cc:107] Waiting for socket to connect
0806 18:37:53.330947 (+ 47us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.194.62:36421: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":38}
W20260806 18:37:53.331211 5448 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260806 18:37:53.332880 20232 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJW69tn/build/debug/bin/kudu as pid 5313
I20260806 18:37:53.990483 5452 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:37:53.989791 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.194.62:36421 (local address 127.19.194.1:40869)
0806 18:37:53.990195 (+ 404us) negotiation.cc:107] Waiting for socket to connect
0806 18:37:53.990287 (+ 92us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.194.62:36421: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":306,"thread_start_us":145,"threads_started":1}
I20260806 18:37:54.147260 5453 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:37:54.146705 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.194.62:36421 (local address 127.19.194.2:54567)
0806 18:37:54.147014 (+ 309us) negotiation.cc:107] Waiting for socket to connect
0806 18:37:54.147104 (+ 90us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.194.62:36421: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":225,"thread_start_us":89,"threads_started":1}
I20260806 18:37:54.332679 5454 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:37:54.332050 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.194.62:36421 (local address 127.19.194.3:40197)
0806 18:37:54.332386 (+ 336us) negotiation.cc:107] Waiting for socket to connect
0806 18:37:54.332509 (+ 123us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.194.62:36421: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":217,"thread_start_us":102,"threads_started":1}
I20260806 18:37:54.425660 5455 raft_consensus.cc:482] T 41b7a086ba4043ecb933bd4721a649ba P 4d2a68a0b77a43619f6206aee728280d [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 18:37:54.426002 5455 raft_consensus.cc:504] T 41b7a086ba4043ecb933bd4721a649ba P 4d2a68a0b77a43619f6206aee728280d [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c5a13411dadf4739ba09a5da11f16926" member_type: VOTER last_known_addr { host: "127.19.194.3" port: 35207 } } peers { permanent_uuid: "4d2a68a0b77a43619f6206aee728280d" member_type: VOTER last_known_addr { host: "127.19.194.2" port: 46303 } } peers { permanent_uuid: "5fdb1c79f09d42078b9fadc4624e6d0d" member_type: VOTER last_known_addr { host: "127.19.194.1" port: 36577 } }
I20260806 18:37:54.428256 5455 leader_election.cc:288] T 41b7a086ba4043ecb933bd4721a649ba P 4d2a68a0b77a43619f6206aee728280d [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers c5a13411dadf4739ba09a5da11f16926 (127.19.194.3:35207), 5fdb1c79f09d42078b9fadc4624e6d0d (127.19.194.1:36577)
Aug 06 18:37:54 dist-test-slave-1xrh krb5kdc[3760](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041473, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.194.2@KRBTEST.COM for kudu/127.19.194.1@KRBTEST.COM
Aug 06 18:37:54 dist-test-slave-1xrh krb5kdc[3760](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041473, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.194.2@KRBTEST.COM for kudu/127.19.194.3@KRBTEST.COM
I20260806 18:37:54.452106 5456 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:37:54.428570 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.194.1:36577 (local address 127.19.194.2:37175)
0806 18:37:54.429249 (+ 679us) negotiation.cc:107] Waiting for socket to connect
0806 18:37:54.429294 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0806 18:37:54.429736 (+ 442us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:37:54.431097 (+ 1361us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:37:54.431143 (+ 46us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:37:54.432159 (+ 1016us) client_negotiation.cc:189] Negotiated authn=SASL
0806 18:37:54.433526 (+ 1367us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:37:54.433554 (+ 28us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:37:54.435548 (+ 1994us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:37:54.435555 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:37:54.437127 (+ 1572us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:37:54.437147 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:37:54.439361 (+ 2214us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:37:54.439365 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:37:54.439491 (+ 126us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:37:54.440650 (+ 1159us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 18:37:54.440675 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0806 18:37:54.443111 (+ 2436us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 18:37:54.447178 (+ 4067us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:37:54.447184 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:37:54.447208 (+ 24us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:37:54.447845 (+ 637us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:37:54.448498 (+ 653us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:37:54.448509 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:37:54.448510 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:37:54.448858 (+ 348us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:37:54.450260 (+ 1402us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 18:37:54.450267 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 18:37:54.450688 (+ 421us) client_negotiation.cc:769] Sending connection context
0806 18:37:54.450959 (+ 271us) client_negotiation.cc:240] Negotiation successful
0806 18:37:54.451808 (+ 849us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":541,"mutex_wait_us":568,"spinlock_wait_cycles":32512,"thread_start_us":106,"threads_started":1}
I20260806 18:37:54.452136 5458 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:37:54.429002 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.194.2:43125 (local address 127.19.194.3:35207)
0806 18:37:54.429770 (+ 768us) server_negotiation.cc:206] Beginning negotiation
0806 18:37:54.429823 (+ 53us) server_negotiation.cc:399] Waiting for connection header
0806 18:37:54.429895 (+ 72us) server_negotiation.cc:407] Connection header received
0806 18:37:54.431266 (+ 1371us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:37:54.431283 (+ 17us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:37:54.432039 (+ 756us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:37:54.432768 (+ 729us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:37:54.433815 (+ 1047us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:37:54.435363 (+ 1548us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:37:54.437381 (+ 2018us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:37:54.438803 (+ 1422us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:37:54.439043 (+ 240us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:37:54.444526 (+ 5483us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:37:54.444556 (+ 30us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:37:54.444618 (+ 62us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 18:37:54.444678 (+ 60us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:37:54.447909 (+ 3231us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:37:54.448676 (+ 767us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:37:54.448693 (+ 17us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:37:54.448698 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:37:54.448965 (+ 267us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:37:54.449498 (+ 533us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:37:54.449501 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:37:54.449503 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:37:54.450323 (+ 820us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:37:54.450510 (+ 187us) server_negotiation.cc:1035] Waiting for connection context
0806 18:37:54.451272 (+ 762us) server_negotiation.cc:299] Negotiation successful
0806 18:37:54.451827 (+ 555us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":534,"thread_start_us":124,"threads_started":1}
I20260806 18:37:54.452126 5457 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:37:54.428853 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.194.2:37175 (local address 127.19.194.1:36577)
0806 18:37:54.429602 (+ 749us) server_negotiation.cc:206] Beginning negotiation
0806 18:37:54.429622 (+ 20us) server_negotiation.cc:399] Waiting for connection header
0806 18:37:54.429680 (+ 58us) server_negotiation.cc:407] Connection header received
0806 18:37:54.430251 (+ 571us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:37:54.430264 (+ 13us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:37:54.430603 (+ 339us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:37:54.430849 (+ 246us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:37:54.433794 (+ 2945us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:37:54.435337 (+ 1543us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:37:54.437334 (+ 1997us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:37:54.438778 (+ 1444us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:37:54.439179 (+ 401us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:37:54.443520 (+ 4341us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:37:54.443541 (+ 21us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:37:54.443571 (+ 30us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 18:37:54.443640 (+ 69us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:37:54.447002 (+ 3362us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:37:54.448054 (+ 1052us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:37:54.448061 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:37:54.448067 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:37:54.448326 (+ 259us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:37:54.449088 (+ 762us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:37:54.449092 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:37:54.449094 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:37:54.450074 (+ 980us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:37:54.450323 (+ 249us) server_negotiation.cc:1035] Waiting for connection context
0806 18:37:54.451272 (+ 949us) server_negotiation.cc:299] Negotiation successful
0806 18:37:54.451820 (+ 548us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":593,"thread_start_us":149,"threads_started":1}
I20260806 18:37:54.452123 5453 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:37:54.428539 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.194.3:35207 (local address 127.19.194.2:43125)
0806 18:37:54.428751 (+ 212us) negotiation.cc:107] Waiting for socket to connect
0806 18:37:54.428802 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0806 18:37:54.429727 (+ 925us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:37:54.432658 (+ 2931us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:37:54.432661 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:37:54.433192 (+ 531us) client_negotiation.cc:189] Negotiated authn=SASL
0806 18:37:54.433525 (+ 333us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:37:54.433552 (+ 27us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:37:54.435560 (+ 2008us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:37:54.435563 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:37:54.437133 (+ 1570us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:37:54.437159 (+ 26us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:37:54.439028 (+ 1869us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:37:54.439034 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:37:54.439438 (+ 404us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:37:54.440524 (+ 1086us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 18:37:54.440561 (+ 37us) client_negotiation.cc:656] calling sasl_client_start()
0806 18:37:54.444227 (+ 3666us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 18:37:54.448054 (+ 3827us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:37:54.448057 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:37:54.448059 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:37:54.448521 (+ 462us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:37:54.449129 (+ 608us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:37:54.449141 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:37:54.449143 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:37:54.449318 (+ 175us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:37:54.450512 (+ 1194us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 18:37:54.450526 (+ 14us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 18:37:54.450688 (+ 162us) client_negotiation.cc:769] Sending connection context
0806 18:37:54.450964 (+ 276us) client_negotiation.cc:240] Negotiation successful
0806 18:37:54.451731 (+ 767us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":78,"mutex_wait_us":190,"spinlock_wait_cycles":24960}
I20260806 18:37:54.454147 5119 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "41b7a086ba4043ecb933bd4721a649ba" candidate_uuid: "4d2a68a0b77a43619f6206aee728280d" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "5fdb1c79f09d42078b9fadc4624e6d0d" is_pre_election: true
I20260806 18:37:54.454660 5119 raft_consensus.cc:2454] T 41b7a086ba4043ecb933bd4721a649ba P 5fdb1c79f09d42078b9fadc4624e6d0d [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4d2a68a0b77a43619f6206aee728280d in term 1.
I20260806 18:37:54.456880 5190 leader_election.cc:302] T 41b7a086ba4043ecb933bd4721a649ba P 4d2a68a0b77a43619f6206aee728280d [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4d2a68a0b77a43619f6206aee728280d, 5fdb1c79f09d42078b9fadc4624e6d0d; no voters:
W20260806 18:37:54.457214 5191 leader_election.cc:341] T 41b7a086ba4043ecb933bd4721a649ba P 4d2a68a0b77a43619f6206aee728280d [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer c5a13411dadf4739ba09a5da11f16926 (127.19.194.3:35207): Not found: Tablet not found: 41b7a086ba4043ecb933bd4721a649ba
I20260806 18:37:54.457350 5455 raft_consensus.cc:2781] T 41b7a086ba4043ecb933bd4721a649ba P 4d2a68a0b77a43619f6206aee728280d [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260806 18:37:54.457571 5455 raft_consensus.cc:482] T 41b7a086ba4043ecb933bd4721a649ba P 4d2a68a0b77a43619f6206aee728280d [term 1 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 18:37:54.457708 5455 raft_consensus.cc:3037] T 41b7a086ba4043ecb933bd4721a649ba P 4d2a68a0b77a43619f6206aee728280d [term 1 FOLLOWER]: Advancing to term 2
I20260806 18:37:54.462397 5455 raft_consensus.cc:504] T 41b7a086ba4043ecb933bd4721a649ba P 4d2a68a0b77a43619f6206aee728280d [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c5a13411dadf4739ba09a5da11f16926" member_type: VOTER last_known_addr { host: "127.19.194.3" port: 35207 } } peers { permanent_uuid: "4d2a68a0b77a43619f6206aee728280d" member_type: VOTER last_known_addr { host: "127.19.194.2" port: 46303 } } peers { permanent_uuid: "5fdb1c79f09d42078b9fadc4624e6d0d" member_type: VOTER last_known_addr { host: "127.19.194.1" port: 36577 } }
I20260806 18:37:54.463063 5455 leader_election.cc:288] T 41b7a086ba4043ecb933bd4721a649ba P 4d2a68a0b77a43619f6206aee728280d [CANDIDATE]: Term 2 election: Requested vote from peers c5a13411dadf4739ba09a5da11f16926 (127.19.194.3:35207), 5fdb1c79f09d42078b9fadc4624e6d0d (127.19.194.1:36577)
I20260806 18:37:54.463660 5119 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "41b7a086ba4043ecb933bd4721a649ba" candidate_uuid: "4d2a68a0b77a43619f6206aee728280d" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "5fdb1c79f09d42078b9fadc4624e6d0d"
I20260806 18:37:54.463969 5119 raft_consensus.cc:3037] T 41b7a086ba4043ecb933bd4721a649ba P 5fdb1c79f09d42078b9fadc4624e6d0d [term 1 FOLLOWER]: Advancing to term 2
W20260806 18:37:54.464138 5191 leader_election.cc:341] T 41b7a086ba4043ecb933bd4721a649ba P 4d2a68a0b77a43619f6206aee728280d [CANDIDATE]: Term 2 election: Tablet error from VoteRequest() call to peer c5a13411dadf4739ba09a5da11f16926 (127.19.194.3:35207): Not found: Tablet not found: 41b7a086ba4043ecb933bd4721a649ba
I20260806 18:37:54.467700 5119 raft_consensus.cc:2454] T 41b7a086ba4043ecb933bd4721a649ba P 5fdb1c79f09d42078b9fadc4624e6d0d [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4d2a68a0b77a43619f6206aee728280d in term 2.
I20260806 18:37:54.468181 5190 leader_election.cc:302] T 41b7a086ba4043ecb933bd4721a649ba P 4d2a68a0b77a43619f6206aee728280d [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4d2a68a0b77a43619f6206aee728280d, 5fdb1c79f09d42078b9fadc4624e6d0d; no voters: c5a13411dadf4739ba09a5da11f16926
I20260806 18:37:54.468451 5455 raft_consensus.cc:2781] T 41b7a086ba4043ecb933bd4721a649ba P 4d2a68a0b77a43619f6206aee728280d [term 2 FOLLOWER]: Leader election won for term 2
I20260806 18:37:54.468726 5455 raft_consensus.cc:686] T 41b7a086ba4043ecb933bd4721a649ba P 4d2a68a0b77a43619f6206aee728280d [term 2 LEADER]: Becoming Leader. State: Replica: 4d2a68a0b77a43619f6206aee728280d, State: Running, Role: LEADER
I20260806 18:37:54.469183 5455 consensus_queue.cc:237] T 41b7a086ba4043ecb933bd4721a649ba P 4d2a68a0b77a43619f6206aee728280d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c5a13411dadf4739ba09a5da11f16926" member_type: VOTER last_known_addr { host: "127.19.194.3" port: 35207 } } peers { permanent_uuid: "4d2a68a0b77a43619f6206aee728280d" member_type: VOTER last_known_addr { host: "127.19.194.2" port: 46303 } } peers { permanent_uuid: "5fdb1c79f09d42078b9fadc4624e6d0d" member_type: VOTER last_known_addr { host: "127.19.194.1" port: 36577 } }
I20260806 18:37:54.469803 5453 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:37:54.469408 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.194.62:36421 (local address 127.19.194.2:53495)
0806 18:37:54.469611 (+ 203us) negotiation.cc:107] Waiting for socket to connect
0806 18:37:54.469677 (+ 66us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.194.62:36421: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":133}
I20260806 18:37:54.918409 5119 raft_consensus.cc:1261] T 41b7a086ba4043ecb933bd4721a649ba P 5fdb1c79f09d42078b9fadc4624e6d0d [term 2 FOLLOWER]: Refusing update from remote peer 4d2a68a0b77a43619f6206aee728280d: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 2 index: 3. (index mismatch)
I20260806 18:37:54.919181 5455 consensus_queue.cc:1035] T 41b7a086ba4043ecb933bd4721a649ba P 4d2a68a0b77a43619f6206aee728280d [LEADER]: Connected to new peer: Peer: permanent_uuid: "5fdb1c79f09d42078b9fadc4624e6d0d" member_type: VOTER last_known_addr { host: "127.19.194.1" port: 36577 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260806 18:37:54.921371 5462 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:37:54.920923 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.194.62:36421 (local address 127.19.194.1:37649)
0806 18:37:54.921182 (+ 259us) negotiation.cc:107] Waiting for socket to connect
0806 18:37:54.921251 (+ 69us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.194.62:36421: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":185,"thread_start_us":57,"threads_started":1}
I20260806 18:37:54.923749 5453 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:37:54.923375 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.194.62:36421 (local address 127.19.194.2:41321)
0806 18:37:54.923616 (+ 241us) negotiation.cc:107] Waiting for socket to connect
0806 18:37:54.923673 (+ 57us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.194.62:36421: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":163}
W20260806 18:37:54.923909 5191 consensus_peers.cc:487] T 41b7a086ba4043ecb933bd4721a649ba P 4d2a68a0b77a43619f6206aee728280d -> Peer c5a13411dadf4739ba09a5da11f16926 (127.19.194.3:35207): Couldn't send request to peer c5a13411dadf4739ba09a5da11f16926. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 41b7a086ba4043ecb933bd4721a649ba. This is attempt 1: this message will repeat every 5th retry.
I20260806 18:37:54.923977 5462 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:37:54.923697 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.194.62:36421 (local address 127.19.194.1:35995)
0806 18:37:54.923858 (+ 161us) negotiation.cc:107] Waiting for socket to connect
0806 18:37:54.923912 (+ 54us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.194.62:36421: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":97}
I20260806 18:37:55.305437 5468 ts_tablet_manager.cc:927] T 41b7a086ba4043ecb933bd4721a649ba P c5a13411dadf4739ba09a5da11f16926: Initiating tablet copy from peer 4d2a68a0b77a43619f6206aee728280d (127.19.194.2:46303)
I20260806 18:37:55.306140 5468 tablet_copy_client.cc:316] T 41b7a086ba4043ecb933bd4721a649ba P c5a13411dadf4739ba09a5da11f16926: tablet copy: Beginning tablet copy session from remote peer at address 127.19.194.2:46303
Aug 06 18:37:55 dist-test-slave-1xrh krb5kdc[3760](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041473, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.194.3@KRBTEST.COM for kudu/127.19.194.2@KRBTEST.COM
I20260806 18:37:55.317798 5470 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:37:55.306658 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.194.2:46303 (local address 127.19.194.3:43157)
0806 18:37:55.306949 (+ 291us) negotiation.cc:107] Waiting for socket to connect
0806 18:37:55.306968 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0806 18:37:55.307084 (+ 116us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:37:55.307546 (+ 462us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:37:55.307552 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:37:55.307969 (+ 417us) client_negotiation.cc:189] Negotiated authn=SASL
0806 18:37:55.308105 (+ 136us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:37:55.308111 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:37:55.309061 (+ 950us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:37:55.309063 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:37:55.309812 (+ 749us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:37:55.309825 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:37:55.310740 (+ 915us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:37:55.310746 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:37:55.310915 (+ 169us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:37:55.311590 (+ 675us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 18:37:55.311603 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0806 18:37:55.313284 (+ 1681us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 18:37:55.315656 (+ 2372us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:37:55.315667 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:37:55.315675 (+ 8us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:37:55.315997 (+ 322us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:37:55.316318 (+ 321us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:37:55.316321 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:37:55.316323 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:37:55.316463 (+ 140us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:37:55.317084 (+ 621us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 18:37:55.317091 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 18:37:55.317246 (+ 155us) client_negotiation.cc:769] Sending connection context
0806 18:37:55.317408 (+ 162us) client_negotiation.cc:240] Negotiation successful
0806 18:37:55.317590 (+ 182us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":228,"thread_start_us":84,"threads_started":1}
I20260806 18:37:55.317798 5471 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:37:55.306792 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.194.3:43157 (local address 127.19.194.2:46303)
0806 18:37:55.307109 (+ 317us) server_negotiation.cc:206] Beginning negotiation
0806 18:37:55.307115 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 18:37:55.307182 (+ 67us) server_negotiation.cc:407] Connection header received
0806 18:37:55.307235 (+ 53us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:37:55.307240 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:37:55.307408 (+ 168us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:37:55.307527 (+ 119us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:37:55.308223 (+ 696us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:37:55.308966 (+ 743us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:37:55.309972 (+ 1006us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:37:55.310590 (+ 618us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:37:55.310729 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:37:55.313424 (+ 2695us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:37:55.313435 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:37:55.313449 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 18:37:55.313478 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:37:55.315536 (+ 2058us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:37:55.316129 (+ 593us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:37:55.316135 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:37:55.316137 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:37:55.316230 (+ 93us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:37:55.316583 (+ 353us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:37:55.316586 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:37:55.316588 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:37:55.316952 (+ 364us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:37:55.317088 (+ 136us) server_negotiation.cc:1035] Waiting for connection context
0806 18:37:55.317494 (+ 406us) server_negotiation.cc:299] Negotiation successful
0806 18:37:55.317635 (+ 141us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":236,"thread_start_us":66,"threads_started":1}
I20260806 18:37:55.319398 5283 tablet_copy_service.cc:140] P 4d2a68a0b77a43619f6206aee728280d: Received BeginTabletCopySession request for tablet 41b7a086ba4043ecb933bd4721a649ba from peer c5a13411dadf4739ba09a5da11f16926 ({username='kudu', principal='kudu/127.19.194.3@KRBTEST.COM'} at 127.19.194.3:43157)
I20260806 18:37:55.319602 5283 tablet_copy_service.cc:161] P 4d2a68a0b77a43619f6206aee728280d: Beginning new tablet copy session on tablet 41b7a086ba4043ecb933bd4721a649ba from peer c5a13411dadf4739ba09a5da11f16926 at {username='kudu', principal='kudu/127.19.194.3@KRBTEST.COM'} at 127.19.194.3:43157: session id = c5a13411dadf4739ba09a5da11f16926-41b7a086ba4043ecb933bd4721a649ba
I20260806 18:37:55.320806 5283 tablet_copy_source_session.cc:205] T 41b7a086ba4043ecb933bd4721a649ba P 4d2a68a0b77a43619f6206aee728280d: Tablet Copy: opened 0 blocks and 1 log segments
I20260806 18:37:55.324708 5468 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 41b7a086ba4043ecb933bd4721a649ba. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 18:37:55.330910 5468 tablet_copy_client.cc:791] T 41b7a086ba4043ecb933bd4721a649ba P c5a13411dadf4739ba09a5da11f16926: tablet copy: Starting download of 0 data blocks...
I20260806 18:37:55.331177 5468 tablet_copy_client.cc:655] T 41b7a086ba4043ecb933bd4721a649ba P c5a13411dadf4739ba09a5da11f16926: tablet copy: Starting download of 1 WAL segments...
I20260806 18:37:55.332916 5468 tablet_copy_client.cc:531] T 41b7a086ba4043ecb933bd4721a649ba P c5a13411dadf4739ba09a5da11f16926: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260806 18:37:55.333601 5470 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:37:55.333374 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.194.62:36421 (local address 127.19.194.3:51913)
0806 18:37:55.333490 (+ 116us) negotiation.cc:107] Waiting for socket to connect
0806 18:37:55.333544 (+ 54us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.194.62:36421: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":50}
I20260806 18:37:55.334544 5468 tablet_bootstrap.cc:492] T 41b7a086ba4043ecb933bd4721a649ba P c5a13411dadf4739ba09a5da11f16926: Bootstrap starting.
I20260806 18:37:55.339778 5468 log.cc:824] T 41b7a086ba4043ecb933bd4721a649ba P c5a13411dadf4739ba09a5da11f16926: Log is configured to *not* fsync() on all Append() calls
I20260806 18:37:55.345800 5468 tablet_bootstrap.cc:492] T 41b7a086ba4043ecb933bd4721a649ba P c5a13411dadf4739ba09a5da11f16926: Bootstrap replayed 1/1 log segments. Stats: ops{read=3 overwritten=0 applied=3 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260806 18:37:55.346084 5468 tablet_bootstrap.cc:492] T 41b7a086ba4043ecb933bd4721a649ba P c5a13411dadf4739ba09a5da11f16926: Bootstrap complete.
I20260806 18:37:55.346374 5468 ts_tablet_manager.cc:1397] T 41b7a086ba4043ecb933bd4721a649ba P c5a13411dadf4739ba09a5da11f16926: Time spent bootstrapping tablet: real 0.012s user 0.010s sys 0.000s
I20260806 18:37:55.348726 5468 raft_consensus.cc:348] T 41b7a086ba4043ecb933bd4721a649ba P c5a13411dadf4739ba09a5da11f16926 [term 2 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c5a13411dadf4739ba09a5da11f16926" member_type: VOTER last_known_addr { host: "127.19.194.3" port: 35207 } } peers { permanent_uuid: "4d2a68a0b77a43619f6206aee728280d" member_type: VOTER last_known_addr { host: "127.19.194.2" port: 46303 } } peers { permanent_uuid: "5fdb1c79f09d42078b9fadc4624e6d0d" member_type: VOTER last_known_addr { host: "127.19.194.1" port: 36577 } }
I20260806 18:37:55.348964 5468 raft_consensus.cc:729] T 41b7a086ba4043ecb933bd4721a649ba P c5a13411dadf4739ba09a5da11f16926 [term 2 FOLLOWER]: Becoming Follower/Learner. State: Replica: c5a13411dadf4739ba09a5da11f16926, State: Initialized, Role: FOLLOWER
I20260806 18:37:55.349365 5468 consensus_queue.cc:260] T 41b7a086ba4043ecb933bd4721a649ba P c5a13411dadf4739ba09a5da11f16926 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 3, Last appended: 2.3, Last appended by leader: 3, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c5a13411dadf4739ba09a5da11f16926" member_type: VOTER last_known_addr { host: "127.19.194.3" port: 35207 } } peers { permanent_uuid: "4d2a68a0b77a43619f6206aee728280d" member_type: VOTER last_known_addr { host: "127.19.194.2" port: 46303 } } peers { permanent_uuid: "5fdb1c79f09d42078b9fadc4624e6d0d" member_type: VOTER last_known_addr { host: "127.19.194.1" port: 36577 } }
I20260806 18:37:55.350251 5468 ts_tablet_manager.cc:1428] T 41b7a086ba4043ecb933bd4721a649ba P c5a13411dadf4739ba09a5da11f16926: Time spent starting tablet: real 0.004s user 0.006s sys 0.000s
I20260806 18:37:55.350463 5470 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:37:55.350205 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.194.62:36421 (local address 127.19.194.3:42949)
0806 18:37:55.350320 (+ 115us) negotiation.cc:107] Waiting for socket to connect
0806 18:37:55.350400 (+ 80us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.194.62:36421: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":46}
I20260806 18:37:55.350975 5283 tablet_copy_service.cc:342] P 4d2a68a0b77a43619f6206aee728280d: Request end of tablet copy session c5a13411dadf4739ba09a5da11f16926-41b7a086ba4043ecb933bd4721a649ba received from {username='kudu', principal='kudu/127.19.194.3@KRBTEST.COM'} at 127.19.194.3:43157
I20260806 18:37:55.351089 5283 tablet_copy_service.cc:433] P 4d2a68a0b77a43619f6206aee728280d: ending tablet copy session c5a13411dadf4739ba09a5da11f16926-41b7a086ba4043ecb933bd4721a649ba on tablet 41b7a086ba4043ecb933bd4721a649ba with peer c5a13411dadf4739ba09a5da11f16926
I20260806 18:37:55.351263 5470 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:37:55.350969 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.194.62:36421 (local address 127.19.194.3:34813)
0806 18:37:55.351094 (+ 125us) negotiation.cc:107] Waiting for socket to connect
0806 18:37:55.351187 (+ 93us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.194.62:36421: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":43}
I20260806 18:37:55.808122 5476 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:37:55.807627 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.194.62:36421 (local address 127.19.194.2:55105)
0806 18:37:55.807943 (+ 316us) negotiation.cc:107] Waiting for socket to connect
0806 18:37:55.807996 (+ 53us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.194.62:36421: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":242,"thread_start_us":79,"threads_started":1}
I20260806 18:37:55.808095 5402 raft_consensus.cc:1203] T 41b7a086ba4043ecb933bd4721a649ba P c5a13411dadf4739ba09a5da11f16926 [term 2 FOLLOWER]: Deduplicated request from leader. Original: 1.2->[2.3-2.3] Dedup: 2.3->[]
I20260806 18:37:55.809109 5470 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:37:55.808894 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.194.62:36421 (local address 127.19.194.3:46079)
0806 18:37:55.808999 (+ 105us) negotiation.cc:107] Waiting for socket to connect
0806 18:37:55.809047 (+ 48us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.194.62:36421: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":43}
I20260806 18:37:55.925019 5477 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:37:55.924635 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.194.62:36421 (local address 127.19.194.1:38001)
0806 18:37:55.924862 (+ 227us) negotiation.cc:107] Waiting for socket to connect
0806 18:37:55.924919 (+ 57us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.194.62:36421: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":159,"thread_start_us":70,"threads_started":1}
I20260806 18:37:56.809156 5478 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:37:56.808705 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.194.62:36421 (local address 127.19.194.2:59033)
0806 18:37:56.808963 (+ 258us) negotiation.cc:107] Waiting for socket to connect
0806 18:37:56.809039 (+ 76us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.194.62:36421: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":176,"thread_start_us":95,"threads_started":1}
I20260806 18:37:56.810158 5479 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:37:56.809735 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.194.62:36421 (local address 127.19.194.3:47235)
0806 18:37:56.810012 (+ 277us) negotiation.cc:107] Waiting for socket to connect
0806 18:37:56.810063 (+ 51us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.194.62:36421: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":213,"thread_start_us":61,"threads_started":1}
I20260806 18:37:56.926175 5480 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:37:56.925742 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.194.62:36421 (local address 127.19.194.1:38739)
0806 18:37:56.926024 (+ 282us) negotiation.cc:107] Waiting for socket to connect
0806 18:37:56.926079 (+ 55us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.194.62:36421: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":211,"thread_start_us":74,"threads_started":1}
I20260806 18:37:57.810205 5481 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:37:57.809825 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.194.62:36421 (local address 127.19.194.2:54449)
0806 18:37:57.810057 (+ 232us) negotiation.cc:107] Waiting for socket to connect
0806 18:37:57.810111 (+ 54us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.194.62:36421: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":166,"thread_start_us":76,"threads_started":1}
I20260806 18:37:57.811115 5482 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:37:57.810760 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.194.62:36421 (local address 127.19.194.3:57723)
0806 18:37:57.810986 (+ 226us) negotiation.cc:107] Waiting for socket to connect
0806 18:37:57.811037 (+ 51us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.194.62:36421: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":150,"thread_start_us":62,"threads_started":1}
I20260806 18:37:57.927390 5483 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:37:57.926945 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.194.62:36421 (local address 127.19.194.1:45077)
0806 18:37:57.927223 (+ 278us) negotiation.cc:107] Waiting for socket to connect
0806 18:37:57.927276 (+ 53us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.194.62:36421: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":197,"thread_start_us":83,"threads_started":1}
I20260806 18:37:58.811481 5484 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:37:58.811006 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.194.62:36421 (local address 127.19.194.2:45497)
0806 18:37:58.811318 (+ 312us) negotiation.cc:107] Waiting for socket to connect
0806 18:37:58.811374 (+ 56us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.194.62:36421: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":237,"thread_start_us":75,"threads_started":1}
I20260806 18:37:58.812222 5484 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:37:58.811986 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.194.62:36421 (local address 127.19.194.2:40699)
0806 18:37:58.812109 (+ 123us) negotiation.cc:107] Waiting for socket to connect
0806 18:37:58.812163 (+ 54us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.194.62:36421: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":50}
I20260806 18:37:58.812438 5485 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:37:58.812020 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.194.62:36421 (local address 127.19.194.3:34631)
0806 18:37:58.812246 (+ 226us) negotiation.cc:107] Waiting for socket to connect
0806 18:37:58.812328 (+ 82us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.194.62:36421: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":147,"thread_start_us":80,"threads_started":1}
I20260806 18:37:58.928869 5486 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:37:58.928378 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.194.62:36421 (local address 127.19.194.1:35177)
0806 18:37:58.928718 (+ 340us) negotiation.cc:107] Waiting for socket to connect
0806 18:37:58.928773 (+ 55us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.194.62:36421: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":266,"thread_start_us":83,"threads_started":1}
I20260806 18:37:59.813292 5487 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:37:59.812859 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.194.62:36421 (local address 127.19.194.2:50609)
0806 18:37:59.813139 (+ 280us) negotiation.cc:107] Waiting for socket to connect
0806 18:37:59.813193 (+ 54us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.194.62:36421: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":199,"thread_start_us":111,"threads_started":1}
I20260806 18:37:59.813545 5488 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:37:59.813097 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.194.62:36421 (local address 127.19.194.3:55041)
0806 18:37:59.813346 (+ 249us) negotiation.cc:107] Waiting for socket to connect
0806 18:37:59.813438 (+ 92us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.194.62:36421: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":170,"thread_start_us":98,"threads_started":1}
I20260806 18:37:59.930011 5489 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:37:59.929603 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.194.62:36421 (local address 127.19.194.1:48799)
0806 18:37:59.929861 (+ 258us) negotiation.cc:107] Waiting for socket to connect
0806 18:37:59.929916 (+ 55us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.194.62:36421: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":193,"thread_start_us":100,"threads_started":1}
I20260806 18:38:00.814590 5490 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:38:00.814137 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.194.62:36421 (local address 127.19.194.2:41173)
0806 18:38:00.814437 (+ 300us) negotiation.cc:107] Waiting for socket to connect
0806 18:38:00.814494 (+ 57us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.194.62:36421: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":228,"thread_start_us":115,"threads_started":1}
I20260806 18:38:00.814679 5491 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:38:00.814254 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.194.62:36421 (local address 127.19.194.3:37169)
0806 18:38:00.814523 (+ 269us) negotiation.cc:107] Waiting for socket to connect
0806 18:38:00.814573 (+ 50us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.194.62:36421: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":194,"thread_start_us":60,"threads_started":1}
I20260806 18:38:00.931082 5492 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:38:00.930687 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.194.62:36421 (local address 127.19.194.1:44819)
0806 18:38:00.930928 (+ 241us) negotiation.cc:107] Waiting for socket to connect
0806 18:38:00.930983 (+ 55us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.194.62:36421: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":170,"thread_start_us":72,"threads_started":1}
I20260806 18:38:01.815943 5493 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:38:01.815484 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.194.62:36421 (local address 127.19.194.3:57067)
0806 18:38:01.815757 (+ 273us) negotiation.cc:107] Waiting for socket to connect
0806 18:38:01.815812 (+ 55us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.194.62:36421: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":202,"thread_start_us":93,"threads_started":1}
I20260806 18:38:01.815964 5494 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:38:01.815493 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.194.62:36421 (local address 127.19.194.2:49551)
0806 18:38:01.815826 (+ 333us) negotiation.cc:107] Waiting for socket to connect
0806 18:38:01.815877 (+ 51us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.194.62:36421: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":274,"thread_start_us":95,"threads_started":1}
I20260806 18:38:01.932361 5495 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:38:01.931881 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.194.62:36421 (local address 127.19.194.1:35649)
0806 18:38:01.932177 (+ 296us) negotiation.cc:107] Waiting for socket to connect
0806 18:38:01.932255 (+ 78us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.194.62:36421: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":190,"thread_start_us":92,"threads_started":1}
I20260806 18:38:02.817117 5496 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:38:02.816695 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.194.62:36421 (local address 127.19.194.2:59959)
0806 18:38:02.816969 (+ 274us) negotiation.cc:107] Waiting for socket to connect
0806 18:38:02.817023 (+ 54us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.194.62:36421: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":194,"thread_start_us":111,"threads_started":1}
I20260806 18:38:02.817221 5497 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:38:02.816813 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.194.62:36421 (local address 127.19.194.3:49181)
0806 18:38:02.817057 (+ 244us) negotiation.cc:107] Waiting for socket to connect
0806 18:38:02.817107 (+ 50us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.194.62:36421: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":166,"thread_start_us":87,"threads_started":1}
I20260806 18:38:02.933475 5498 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:38:02.933073 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.194.62:36421 (local address 127.19.194.1:57709)
0806 18:38:02.933314 (+ 241us) negotiation.cc:107] Waiting for socket to connect
0806 18:38:02.933369 (+ 55us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.194.62:36421: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":174,"thread_start_us":72,"threads_started":1}
I20260806 18:38:03.818486 5499 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:38:03.818028 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.194.62:36421 (local address 127.19.194.2:60779)
0806 18:38:03.818307 (+ 279us) negotiation.cc:107] Waiting for socket to connect
0806 18:38:03.818373 (+ 66us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.194.62:36421: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":198,"thread_start_us":73,"threads_started":1}
I20260806 18:38:03.818519 5500 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:38:03.818070 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.194.62:36421 (local address 127.19.194.3:34199)
0806 18:38:03.818362 (+ 292us) negotiation.cc:107] Waiting for socket to connect
0806 18:38:03.818431 (+ 69us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.194.62:36421: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":219,"thread_start_us":112,"threads_started":1}
I20260806 18:38:03.934692 5501 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:38:03.934302 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.194.62:36421 (local address 127.19.194.1:32825)
0806 18:38:03.934540 (+ 238us) negotiation.cc:107] Waiting for socket to connect
0806 18:38:03.934593 (+ 53us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.194.62:36421: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":173,"thread_start_us":79,"threads_started":1}
I20260806 18:38:04.819799 5502 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:38:04.819299 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.194.62:36421 (local address 127.19.194.2:45413)
0806 18:38:04.819650 (+ 351us) negotiation.cc:107] Waiting for socket to connect
0806 18:38:04.819702 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.194.62:36421: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":273,"thread_start_us":96,"threads_started":1}
I20260806 18:38:04.819829 5503 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:38:04.819331 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.194.62:36421 (local address 127.19.194.3:60363)
0806 18:38:04.819688 (+ 357us) negotiation.cc:107] Waiting for socket to connect
0806 18:38:04.819742 (+ 54us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.194.62:36421: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":287,"thread_start_us":112,"threads_started":1}
I20260806 18:38:04.935880 5504 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:38:04.935476 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.194.62:36421 (local address 127.19.194.1:48823)
0806 18:38:04.935728 (+ 252us) negotiation.cc:107] Waiting for socket to connect
0806 18:38:04.935783 (+ 55us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.194.62:36421: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":182,"thread_start_us":77,"threads_started":1}
I20260806 18:38:05.821027 5505 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:38:05.820554 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.194.62:36421 (local address 127.19.194.2:58295)
0806 18:38:05.820838 (+ 284us) negotiation.cc:107] Waiting for socket to connect
0806 18:38:05.820903 (+ 65us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.194.62:36421: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":200,"thread_start_us":79,"threads_started":1}
I20260806 18:38:05.821098 5506 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:38:05.820707 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.194.62:36421 (local address 127.19.194.3:54099)
0806 18:38:05.820954 (+ 247us) negotiation.cc:107] Waiting for socket to connect
0806 18:38:05.821008 (+ 54us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.194.62:36421: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":169,"thread_start_us":74,"threads_started":1}
I20260806 18:38:05.936987 5507 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:38:05.936576 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.194.62:36421 (local address 127.19.194.1:42137)
0806 18:38:05.936818 (+ 242us) negotiation.cc:107] Waiting for socket to connect
0806 18:38:05.936873 (+ 55us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.194.62:36421: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":173,"thread_start_us":74,"threads_started":1}
I20260806 18:38:06.822331 5510 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:38:06.821856 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.194.62:36421 (local address 127.19.194.2:59641)
0806 18:38:06.822146 (+ 290us) negotiation.cc:107] Waiting for socket to connect
0806 18:38:06.822200 (+ 54us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.194.62:36421: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":209,"thread_start_us":96,"threads_started":1}
I20260806 18:38:06.822325 5509 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:38:06.821851 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.194.62:36421 (local address 127.19.194.3:35075)
0806 18:38:06.822144 (+ 293us) negotiation.cc:107] Waiting for socket to connect
0806 18:38:06.822209 (+ 65us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.194.62:36421: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":205,"thread_start_us":117,"threads_started":1}
I20260806 18:38:06.938117 5511 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:38:06.937705 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.194.62:36421 (local address 127.19.194.1:46409)
0806 18:38:06.937957 (+ 252us) negotiation.cc:107] Waiting for socket to connect
0806 18:38:06.938015 (+ 58us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.194.62:36421: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":182,"thread_start_us":76,"threads_started":1}
I20260806 18:38:07.823659 5512 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:38:07.823205 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.194.62:36421 (local address 127.19.194.3:41429)
0806 18:38:07.823495 (+ 290us) negotiation.cc:107] Waiting for socket to connect
0806 18:38:07.823563 (+ 68us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.194.62:36421: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":209,"thread_start_us":82,"threads_started":1}
I20260806 18:38:07.823688 5513 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:38:07.823283 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.194.62:36421 (local address 127.19.194.2:50827)
0806 18:38:07.823551 (+ 268us) negotiation.cc:107] Waiting for socket to connect
0806 18:38:07.823602 (+ 51us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.194.62:36421: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":201,"thread_start_us":78,"threads_started":1}
I20260806 18:38:07.939505 5514 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:38:07.938949 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.194.62:36421 (local address 127.19.194.1:50043)
0806 18:38:07.939266 (+ 317us) negotiation.cc:107] Waiting for socket to connect
0806 18:38:07.939338 (+ 72us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.194.62:36421: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":225,"thread_start_us":84,"threads_started":1}
I20260806 18:38:08.340965 20232 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJW69tn/build/debug/bin/kudu with pid 4981
I20260806 18:38:08.349433 20232 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJW69tn/build/debug/bin/kudu with pid 5170
I20260806 18:38:08.356320 20232 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJW69tn/build/debug/bin/kudu with pid 5313
2026-08-06T18:38:08Z chronyd exiting
[ OK ] SecurityITest.TestEnableAndDisableEncryptedTSK (59236 ms)
[----------] 30 tests from SecurityITest (242258 ms total)
[----------] 6 tests from EncryptionPolicyTest
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/0
Loading random data
Initializing database '/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786041246094603-20232-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 18:38:08 dist-test-slave-1xrh krb5kdc[5517](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 18:38:08 dist-test-slave-1xrh krb5kdc[5517](info): set up 2 sockets
Aug 06 18:38:08 dist-test-slave-1xrh krb5kdc[5517](info): commencing operation
krb5kdc: starting...
W20260806 18:38:09.430135 20232 mini_kdc.cc:121] Time spent starting KDC: real 1.049s user 0.007s sys 0.000s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786041246094603-20232-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786041246094603-20232-0/krb5kdc/test-user.keytab.
Aug 06 18:38:09 dist-test-slave-1xrh krb5kdc[5517](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041489, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T18:38:09Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T18:38:09Z Disabled control of system clock
WARNING: no policy specified for kudu/127.19.194.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.19.194.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.19.194.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786041246094603-20232-0/krb5kdc/kudu_127.19.194.62.keytab.
Entry for principal kudu/127.19.194.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786041246094603-20232-0/krb5kdc/kudu_127.19.194.62.keytab.
I20260806 18:38:09.551209 20232 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJW69tn/build/debug/bin/kudu
/tmp/dist-test-taskJW69tn/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786041246094603-20232-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786041246094603-20232-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786041246094603-20232-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786041246094603-20232-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.19.194.62:43945
--webserver_interface=127.19.194.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786041246094603-20232-0/krb5kdc/kudu_127.19.194.62.keytab
--principal=kudu/127.19.194.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.19.194.20:36137
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.19.194.62:43945
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786041246094603-20232-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786041246094603-20232-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786041246094603-20232-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 18:38:09.659356 5530 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 18:38:09.659600 5530 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 18:38:09.659655 5530 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 18:38:09.663698 5530 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 18:38:09.663797 5530 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 18:38:09.663836 5530 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 18:38:09.663916 5530 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 18:38:09.663952 5530 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 18:38:09.669145 5530 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.194.20:36137
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786041246094603-20232-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786041246094603-20232-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.19.194.62:43945
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786041246094603-20232-0/krb5kdc/kudu_127.19.194.62.keytab
--principal=kudu/127.19.194.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.19.194.62:43945
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786041246094603-20232-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.19.194.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5530
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786041246094603-20232-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 17:59:13 UTC on ccycloud.quasar-wvsnzx.root.comops.site
build id 4571028
I20260806 18:38:09.670141 5530 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 18:38:09.671180 5530 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 18:38:09.679940 5540 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:38:09.679986 5537 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:38:09.680001 5538 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 18:38:09.680397 5530 server_base.cc:1034] running on GCE node
I20260806 18:38:09.680946 5530 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 18:38:09.682049 5530 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 18:38:09.683197 5530 hybrid_clock.cc:648] HybridClock initialized: now 1786041489683160 us; error 42 us; skew 500 ppm
Aug 06 18:38:09 dist-test-slave-1xrh krb5kdc[5517](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041489, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.194.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 18:38:09.685892 5530 init.cc:377] Logged in from keytab as kudu/127.19.194.62@KRBTEST.COM (short username kudu)
I20260806 18:38:09.686802 5530 webserver.cc:460] Webserver started at http://127.19.194.62:34623/ using document root <none> and password file <none>
I20260806 18:38:09.687381 5530 fs_manager.cc:359] Metadata directory not provided
I20260806 18:38:09.687445 5530 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 18:38:09.687610 5530 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 18:38:09.689539 5530 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786041246094603-20232-0/minicluster-data/master-0/data/instance:
uuid: "d0be563f6e7e457a93df7a191b190f66"
format_stamp: "Formatted at 2026-08-06 18:38:09 on dist-test-slave-1xrh"
server_key: "17f3b84e3f1062630579cdce80b5da78"
server_key_iv: "efee9c26dedc180933dca4aea8fa11fd"
server_key_version: "encryptionkey@0"
I20260806 18:38:09.690042 5530 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786041246094603-20232-0/minicluster-data/master-0/wal/instance:
uuid: "d0be563f6e7e457a93df7a191b190f66"
format_stamp: "Formatted at 2026-08-06 18:38:09 on dist-test-slave-1xrh"
server_key: "17f3b84e3f1062630579cdce80b5da78"
server_key_iv: "efee9c26dedc180933dca4aea8fa11fd"
server_key_version: "encryptionkey@0"
I20260806 18:38:09.693697 5530 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.006s sys 0.000s
I20260806 18:38:09.695981 5547 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 18:38:09.697113 5530 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.000s sys 0.003s
I20260806 18:38:09.697253 5530 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786041246094603-20232-0/minicluster-data/master-0/data,/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786041246094603-20232-0/minicluster-data/master-0/wal
uuid: "d0be563f6e7e457a93df7a191b190f66"
format_stamp: "Formatted at 2026-08-06 18:38:09 on dist-test-slave-1xrh"
server_key: "17f3b84e3f1062630579cdce80b5da78"
server_key_iv: "efee9c26dedc180933dca4aea8fa11fd"
server_key_version: "encryptionkey@0"
I20260806 18:38:09.697362 5530 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786041246094603-20232-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786041246094603-20232-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786041246094603-20232-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 18:38:09.713405 5530 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 18:38:09.716306 5530 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 18:38:09.716518 5530 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 18:38:09.723603 5530 rpc_server.cc:307] RPC server started. Bound to: 127.19.194.62:43945
I20260806 18:38:09.723615 5604 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.194.62:43945 every 8 connection(s)
I20260806 18:38:09.724555 5530 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786041246094603-20232-0/minicluster-data/master-0/data/info.pb
I20260806 18:38:09.726749 20232 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJW69tn/build/debug/bin/kudu as pid 5530
I20260806 18:38:09.726872 20232 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786041246094603-20232-0/minicluster-data/master-0/wal/instance
I20260806 18:38:09.727257 20232 external_mini_cluster.cc:1428] Setting key 3dd99264153a48492f53e7e4aa9ff052
I20260806 18:38:09.727383 5605 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 18:38:09.734108 5605 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d0be563f6e7e457a93df7a191b190f66: Bootstrap starting.
Aug 06 18:38:09 dist-test-slave-1xrh krb5kdc[5517](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041489, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.19.194.62@KRBTEST.COM
I20260806 18:38:09.736547 5605 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P d0be563f6e7e457a93df7a191b190f66: Neither blocks nor log segments found. Creating new log.
I20260806 18:38:09.737251 5605 log.cc:824] T 00000000000000000000000000000000 P d0be563f6e7e457a93df7a191b190f66: Log is configured to *not* fsync() on all Append() calls
I20260806 18:38:09.739517 5605 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d0be563f6e7e457a93df7a191b190f66: No bootstrap required, opened a new log
I20260806 18:38:09.742175 5608 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:38:09.728561 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51320 (local address 127.19.194.62:43945)
0806 18:38:09.728941 (+ 380us) server_negotiation.cc:206] Beginning negotiation
0806 18:38:09.728959 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0806 18:38:09.729004 (+ 45us) server_negotiation.cc:407] Connection header received
0806 18:38:09.729939 (+ 935us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:38:09.729957 (+ 18us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:38:09.730256 (+ 299us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:38:09.730519 (+ 263us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:38:09.731462 (+ 943us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:38:09.732430 (+ 968us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:38:09.733617 (+ 1187us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:38:09.734194 (+ 577us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:38:09.734348 (+ 154us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:38:09.737101 (+ 2753us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:38:09.737105 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:38:09.737118 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 18:38:09.737144 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:38:09.739087 (+ 1943us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:38:09.739562 (+ 475us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:38:09.739569 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:38:09.739574 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:38:09.739705 (+ 131us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:38:09.739971 (+ 266us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:38:09.739974 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:38:09.739976 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:38:09.740447 (+ 471us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:38:09.740655 (+ 208us) server_negotiation.cc:1035] Waiting for connection context
0806 18:38:09.740890 (+ 235us) server_negotiation.cc:299] Negotiation successful
0806 18:38:09.741172 (+ 282us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":253,"thread_start_us":140,"threads_started":1}
I20260806 18:38:09.743929 5605 raft_consensus.cc:348] T 00000000000000000000000000000000 P d0be563f6e7e457a93df7a191b190f66 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d0be563f6e7e457a93df7a191b190f66" member_type: VOTER last_known_addr { host: "127.19.194.62" port: 43945 } }
I20260806 18:38:09.744179 5605 raft_consensus.cc:374] T 00000000000000000000000000000000 P d0be563f6e7e457a93df7a191b190f66 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 18:38:09.744294 5605 raft_consensus.cc:729] T 00000000000000000000000000000000 P d0be563f6e7e457a93df7a191b190f66 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d0be563f6e7e457a93df7a191b190f66, State: Initialized, Role: FOLLOWER
I20260806 18:38:09.744688 5605 consensus_queue.cc:260] T 00000000000000000000000000000000 P d0be563f6e7e457a93df7a191b190f66 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d0be563f6e7e457a93df7a191b190f66" member_type: VOTER last_known_addr { host: "127.19.194.62" port: 43945 } }
I20260806 18:38:09.744804 5605 raft_consensus.cc:388] T 00000000000000000000000000000000 P d0be563f6e7e457a93df7a191b190f66 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 18:38:09.744853 5605 raft_consensus.cc:482] T 00000000000000000000000000000000 P d0be563f6e7e457a93df7a191b190f66 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 18:38:09.744927 5605 raft_consensus.cc:3037] T 00000000000000000000000000000000 P d0be563f6e7e457a93df7a191b190f66 [term 0 FOLLOWER]: Advancing to term 1
I20260806 18:38:09.745751 5605 raft_consensus.cc:504] T 00000000000000000000000000000000 P d0be563f6e7e457a93df7a191b190f66 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d0be563f6e7e457a93df7a191b190f66" member_type: VOTER last_known_addr { host: "127.19.194.62" port: 43945 } }
I20260806 18:38:09.746029 5605 leader_election.cc:302] T 00000000000000000000000000000000 P d0be563f6e7e457a93df7a191b190f66 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: d0be563f6e7e457a93df7a191b190f66; no voters:
I20260806 18:38:09.746264 5605 leader_election.cc:288] T 00000000000000000000000000000000 P d0be563f6e7e457a93df7a191b190f66 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 18:38:09.746390 5610 raft_consensus.cc:2781] T 00000000000000000000000000000000 P d0be563f6e7e457a93df7a191b190f66 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 18:38:09.746670 5610 raft_consensus.cc:686] T 00000000000000000000000000000000 P d0be563f6e7e457a93df7a191b190f66 [term 1 LEADER]: Becoming Leader. State: Replica: d0be563f6e7e457a93df7a191b190f66, State: Running, Role: LEADER
I20260806 18:38:09.747005 5610 consensus_queue.cc:237] T 00000000000000000000000000000000 P d0be563f6e7e457a93df7a191b190f66 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d0be563f6e7e457a93df7a191b190f66" member_type: VOTER last_known_addr { host: "127.19.194.62" port: 43945 } }
I20260806 18:38:09.747483 5605 sys_catalog.cc:564] T 00000000000000000000000000000000 P d0be563f6e7e457a93df7a191b190f66 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 18:38:09.748997 5612 sys_catalog.cc:455] T 00000000000000000000000000000000 P d0be563f6e7e457a93df7a191b190f66 [sys.catalog]: SysCatalogTable state changed. Reason: New leader d0be563f6e7e457a93df7a191b190f66. Latest consensus state: current_term: 1 leader_uuid: "d0be563f6e7e457a93df7a191b190f66" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d0be563f6e7e457a93df7a191b190f66" member_type: VOTER last_known_addr { host: "127.19.194.62" port: 43945 } } }
I20260806 18:38:09.749117 5612 sys_catalog.cc:458] T 00000000000000000000000000000000 P d0be563f6e7e457a93df7a191b190f66 [sys.catalog]: This master's current role is: LEADER
I20260806 18:38:09.749521 5611 sys_catalog.cc:455] T 00000000000000000000000000000000 P d0be563f6e7e457a93df7a191b190f66 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "d0be563f6e7e457a93df7a191b190f66" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d0be563f6e7e457a93df7a191b190f66" member_type: VOTER last_known_addr { host: "127.19.194.62" port: 43945 } } }
I20260806 18:38:09.749657 5620 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 18:38:09.749671 5611 sys_catalog.cc:458] T 00000000000000000000000000000000 P d0be563f6e7e457a93df7a191b190f66 [sys.catalog]: This master's current role is: LEADER
I20260806 18:38:09.752125 5620 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 18:38:09.756851 5620 catalog_manager.cc:1347] Generated new cluster ID: 921b5b16fdd94457952724ebd57cafc4
I20260806 18:38:09.756911 5620 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 18:38:09.764957 5620 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 18:38:09.766088 5620 catalog_manager.cc:1504] Loading token signing keys...
I20260806 18:38:09.772210 5620 catalog_manager.cc:5868] T 00000000000000000000000000000000 P d0be563f6e7e457a93df7a191b190f66: Generated new TSK 0
I20260806 18:38:09.772917 5620 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.19.194.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.19.194.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.19.194.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786041246094603-20232-0/krb5kdc/kudu_127.19.194.1.keytab.
Entry for principal kudu/127.19.194.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786041246094603-20232-0/krb5kdc/kudu_127.19.194.1.keytab.
I20260806 18:38:09.806631 20232 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJW69tn/build/debug/bin/kudu
/tmp/dist-test-taskJW69tn/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786041246094603-20232-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786041246094603-20232-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786041246094603-20232-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786041246094603-20232-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.19.194.1:0
--local_ip_for_outbound_sockets=127.19.194.1
--webserver_interface=127.19.194.1
--webserver_port=0
--tserver_master_addrs=127.19.194.62:43945
--keytab_file=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786041246094603-20232-0/krb5kdc/kudu_127.19.194.1.keytab
--principal=kudu/127.19.194.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.19.194.20:36137
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786041246094603-20232-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786041246094603-20232-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786041246094603-20232-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 18:38:09.914198 5631 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 18:38:09.914436 5631 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 18:38:09.914507 5631 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 18:38:09.918507 5631 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 18:38:09.918560 5631 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 18:38:09.918613 5631 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.19.194.1
I20260806 18:38:09.923666 5631 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.194.20:36137
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786041246094603-20232-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786041246094603-20232-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786041246094603-20232-0/krb5kdc/kudu_127.19.194.1.keytab
--principal=kudu/127.19.194.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.19.194.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786041246094603-20232-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.19.194.1
--webserver_port=0
--tserver_master_addrs=127.19.194.62:43945
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5631
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.19.194.1
--log_dir=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786041246094603-20232-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 17:59:13 UTC on ccycloud.quasar-wvsnzx.root.comops.site
build id 4571028
I20260806 18:38:09.924669 5631 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 18:38:09.925634 5631 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 18:38:09.935468 5637 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:38:09.935485 5638 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 18:38:09.935734 5631 server_base.cc:1034] running on GCE node
W20260806 18:38:09.935968 5640 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 18:38:09.936388 5631 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 18:38:09.937027 5631 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 18:38:09.938207 5631 hybrid_clock.cc:648] HybridClock initialized: now 1786041489938187 us; error 32 us; skew 500 ppm
Aug 06 18:38:09 dist-test-slave-1xrh krb5kdc[5517](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041489, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.194.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 18:38:09.941006 5631 init.cc:377] Logged in from keytab as kudu/127.19.194.1@KRBTEST.COM (short username kudu)
I20260806 18:38:09.941854 5631 webserver.cc:460] Webserver started at http://127.19.194.1:44063/ using document root <none> and password file <none>
I20260806 18:38:09.942392 5631 fs_manager.cc:359] Metadata directory not provided
I20260806 18:38:09.942502 5631 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 18:38:09.942742 5631 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 18:38:09.944751 5631 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786041246094603-20232-0/minicluster-data/ts-0/data/instance:
uuid: "8219dd885a9d4d3ab75e45276b98997d"
format_stamp: "Formatted at 2026-08-06 18:38:09 on dist-test-slave-1xrh"
server_key: "66df8fc70289a006d711495d963491ed"
server_key_iv: "409e40ec23ab94bb22e28e5a0b14b4dd"
server_key_version: "encryptionkey@0"
I20260806 18:38:09.945312 5631 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786041246094603-20232-0/minicluster-data/ts-0/wal/instance:
uuid: "8219dd885a9d4d3ab75e45276b98997d"
format_stamp: "Formatted at 2026-08-06 18:38:09 on dist-test-slave-1xrh"
server_key: "66df8fc70289a006d711495d963491ed"
server_key_iv: "409e40ec23ab94bb22e28e5a0b14b4dd"
server_key_version: "encryptionkey@0"
I20260806 18:38:09.949059 5631 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.002s
I20260806 18:38:09.951498 5647 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 18:38:09.952682 5631 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260806 18:38:09.952809 5631 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786041246094603-20232-0/minicluster-data/ts-0/data,/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786041246094603-20232-0/minicluster-data/ts-0/wal
uuid: "8219dd885a9d4d3ab75e45276b98997d"
format_stamp: "Formatted at 2026-08-06 18:38:09 on dist-test-slave-1xrh"
server_key: "66df8fc70289a006d711495d963491ed"
server_key_iv: "409e40ec23ab94bb22e28e5a0b14b4dd"
server_key_version: "encryptionkey@0"
I20260806 18:38:09.952913 5631 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786041246094603-20232-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786041246094603-20232-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786041246094603-20232-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 18:38:09.979848 5631 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 18:38:09.983918 5631 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 18:38:09.984131 5631 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 18:38:09.984768 5631 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 18:38:09.985780 5631 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 18:38:09.985834 5631 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 18:38:09.985921 5631 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 18:38:09.985957 5631 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 18:38:09.995093 5631 rpc_server.cc:307] RPC server started. Bound to: 127.19.194.1:37765
I20260806 18:38:09.995178 5770 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.194.1:37765 every 8 connection(s)
I20260806 18:38:09.996053 5631 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786041246094603-20232-0/minicluster-data/ts-0/data/info.pb
I20260806 18:38:10.002974 20232 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJW69tn/build/debug/bin/kudu as pid 5631
I20260806 18:38:10.003173 20232 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786041246094603-20232-0/minicluster-data/ts-0/wal/instance
I20260806 18:38:10.003521 20232 external_mini_cluster.cc:1428] Setting key 4cf5a5ed28a38a2cfd3b6377bc1ebbc7
Aug 06 18:38:09 dist-test-slave-1xrh krb5kdc[5517](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041489, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.194.1@KRBTEST.COM for kudu/127.19.194.62@KRBTEST.COM
I20260806 18:38:10.010900 5608 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:38:09.997955 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.194.1:51023 (local address 127.19.194.62:43945)
0806 18:38:09.998092 (+ 137us) server_negotiation.cc:206] Beginning negotiation
0806 18:38:09.998096 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 18:38:09.998962 (+ 866us) server_negotiation.cc:407] Connection header received
0806 18:38:10.000467 (+ 1505us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:38:10.000470 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:38:10.000530 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:38:10.000632 (+ 102us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:38:10.001831 (+ 1199us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:38:10.002376 (+ 545us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:38:10.003695 (+ 1319us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:38:10.003978 (+ 283us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:38:10.004101 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:38:10.006940 (+ 2839us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:38:10.006944 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:38:10.006946 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 18:38:10.006971 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:38:10.008629 (+ 1658us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:38:10.009200 (+ 571us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:38:10.009206 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:38:10.009208 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:38:10.009298 (+ 90us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:38:10.009683 (+ 385us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:38:10.009688 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:38:10.009691 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:38:10.009900 (+ 209us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:38:10.010022 (+ 122us) server_negotiation.cc:1035] Waiting for connection context
0806 18:38:10.010552 (+ 530us) server_negotiation.cc:299] Negotiation successful
0806 18:38:10.010695 (+ 143us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260806 18:38:10.011785 5773 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:38:09.998326 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.194.62:43945 (local address 127.19.194.1:51023)
0806 18:38:09.998802 (+ 476us) negotiation.cc:107] Waiting for socket to connect
0806 18:38:09.998854 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0806 18:38:10.000245 (+ 1391us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:38:10.000808 (+ 563us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:38:10.000815 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:38:10.001161 (+ 346us) client_negotiation.cc:189] Negotiated authn=SASL
0806 18:38:10.001684 (+ 523us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:38:10.001697 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:38:10.002524 (+ 827us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:38:10.002527 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:38:10.003544 (+ 1017us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:38:10.003554 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:38:10.004179 (+ 625us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:38:10.004184 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:38:10.004430 (+ 246us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:38:10.005097 (+ 667us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 18:38:10.005115 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0806 18:38:10.006749 (+ 1634us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 18:38:10.008763 (+ 2014us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:38:10.008770 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:38:10.008785 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:38:10.009080 (+ 295us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:38:10.009409 (+ 329us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:38:10.009414 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:38:10.009416 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:38:10.009573 (+ 157us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:38:10.010022 (+ 449us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 18:38:10.010030 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 18:38:10.010284 (+ 254us) client_negotiation.cc:769] Sending connection context
0806 18:38:10.010503 (+ 219us) client_negotiation.cc:240] Negotiation successful
0806 18:38:10.010848 (+ 345us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":269,"thread_start_us":126,"threads_started":1}
I20260806 18:38:10.013140 5771 heartbeater.cc:347] Connected to a master server at 127.19.194.62:43945
WARNING: no policy specified for kudu/127.19.194.2@KRBTEST.COM; defaulting to no policy
I20260806 18:38:10.013410 5771 heartbeater.cc:464] Registering TS with master...
I20260806 18:38:10.013932 5771 heartbeater.cc:511] Master 127.19.194.62:43945 requested a full tablet report, sending...
I20260806 18:38:10.015532 5565 ts_manager.cc:194] Registered new tserver with Master: 8219dd885a9d4d3ab75e45276b98997d (127.19.194.1:37765)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.19.194.2@KRBTEST.COM" created.
I20260806 18:38:10.016815 5565 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.19.194.1@KRBTEST.COM'} at 127.19.194.1:51023
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.19.194.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786041246094603-20232-0/krb5kdc/kudu_127.19.194.2.keytab.
Entry for principal kudu/127.19.194.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786041246094603-20232-0/krb5kdc/kudu_127.19.194.2.keytab.
I20260806 18:38:10.033712 20232 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJW69tn/build/debug/bin/kudu
/tmp/dist-test-taskJW69tn/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786041246094603-20232-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786041246094603-20232-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786041246094603-20232-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786041246094603-20232-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.19.194.2:0
--local_ip_for_outbound_sockets=127.19.194.2
--webserver_interface=127.19.194.2
--webserver_port=0
--tserver_master_addrs=127.19.194.62:43945
--keytab_file=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786041246094603-20232-0/krb5kdc/kudu_127.19.194.2.keytab
--principal=kudu/127.19.194.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.19.194.20:36137
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786041246094603-20232-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786041246094603-20232-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786041246094603-20232-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 18:38:10.140926 5776 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 18:38:10.141149 5776 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 18:38:10.141191 5776 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 18:38:10.145028 5776 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 18:38:10.145073 5776 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 18:38:10.145128 5776 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.19.194.2
I20260806 18:38:10.150310 5776 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.194.20:36137
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786041246094603-20232-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786041246094603-20232-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786041246094603-20232-0/krb5kdc/kudu_127.19.194.2.keytab
--principal=kudu/127.19.194.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.19.194.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786041246094603-20232-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.19.194.2
--webserver_port=0
--tserver_master_addrs=127.19.194.62:43945
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5776
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.19.194.2
--log_dir=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786041246094603-20232-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 17:59:13 UTC on ccycloud.quasar-wvsnzx.root.comops.site
build id 4571028
I20260806 18:38:10.151219 5776 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 18:38:10.152105 5776 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 18:38:10.161466 5781 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:38:10.161438 5782 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 18:38:10.161650 5776 server_base.cc:1034] running on GCE node
W20260806 18:38:10.161450 5784 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 18:38:10.162191 5776 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 18:38:10.162750 5776 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 18:38:10.163904 5776 hybrid_clock.cc:648] HybridClock initialized: now 1786041490163878 us; error 39 us; skew 500 ppm
Aug 06 18:38:10 dist-test-slave-1xrh krb5kdc[5517](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041490, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.194.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 18:38:10.166733 5776 init.cc:377] Logged in from keytab as kudu/127.19.194.2@KRBTEST.COM (short username kudu)
I20260806 18:38:10.167636 5776 webserver.cc:460] Webserver started at http://127.19.194.2:35313/ using document root <none> and password file <none>
I20260806 18:38:10.168154 5776 fs_manager.cc:359] Metadata directory not provided
I20260806 18:38:10.168213 5776 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 18:38:10.168462 5776 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 18:38:10.170384 5776 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786041246094603-20232-0/minicluster-data/ts-1/data/instance:
uuid: "10225ccf5e6d4fb884371ff6c8dd4b9a"
format_stamp: "Formatted at 2026-08-06 18:38:10 on dist-test-slave-1xrh"
server_key: "f974c2a758d4244b85e7a1e674c75191"
server_key_iv: "20b5993396408b01e963cdd69a4407d2"
server_key_version: "encryptionkey@0"
I20260806 18:38:10.170939 5776 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786041246094603-20232-0/minicluster-data/ts-1/wal/instance:
uuid: "10225ccf5e6d4fb884371ff6c8dd4b9a"
format_stamp: "Formatted at 2026-08-06 18:38:10 on dist-test-slave-1xrh"
server_key: "f974c2a758d4244b85e7a1e674c75191"
server_key_iv: "20b5993396408b01e963cdd69a4407d2"
server_key_version: "encryptionkey@0"
I20260806 18:38:10.174450 5776 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.002s sys 0.001s
I20260806 18:38:10.176702 5791 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 18:38:10.177816 5776 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.000s sys 0.001s
I20260806 18:38:10.177954 5776 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786041246094603-20232-0/minicluster-data/ts-1/data,/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786041246094603-20232-0/minicluster-data/ts-1/wal
uuid: "10225ccf5e6d4fb884371ff6c8dd4b9a"
format_stamp: "Formatted at 2026-08-06 18:38:10 on dist-test-slave-1xrh"
server_key: "f974c2a758d4244b85e7a1e674c75191"
server_key_iv: "20b5993396408b01e963cdd69a4407d2"
server_key_version: "encryptionkey@0"
I20260806 18:38:10.178073 5776 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786041246094603-20232-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786041246094603-20232-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786041246094603-20232-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 18:38:10.188419 5776 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 18:38:10.191553 5776 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 18:38:10.191741 5776 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 18:38:10.192278 5776 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 18:38:10.193087 5776 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 18:38:10.193153 5776 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 18:38:10.193213 5776 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 18:38:10.193260 5776 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 18:38:10.201965 5776 rpc_server.cc:307] RPC server started. Bound to: 127.19.194.2:40689
I20260806 18:38:10.202045 5910 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.194.2:40689 every 8 connection(s)
I20260806 18:38:10.202795 5776 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786041246094603-20232-0/minicluster-data/ts-1/data/info.pb
I20260806 18:38:10.209568 20232 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJW69tn/build/debug/bin/kudu as pid 5776
I20260806 18:38:10.209692 20232 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786041246094603-20232-0/minicluster-data/ts-1/wal/instance
I20260806 18:38:10.210002 20232 external_mini_cluster.cc:1428] Setting key d35ee88d72fe0e61afcd8bcc5eed7bbb
Aug 06 18:38:10 dist-test-slave-1xrh krb5kdc[5517](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041490, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.194.2@KRBTEST.COM for kudu/127.19.194.62@KRBTEST.COM
I20260806 18:38:10.217156 5608 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:38:10.204541 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.194.2:46487 (local address 127.19.194.62:43945)
0806 18:38:10.204678 (+ 137us) server_negotiation.cc:206] Beginning negotiation
0806 18:38:10.204683 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 18:38:10.205495 (+ 812us) server_negotiation.cc:407] Connection header received
0806 18:38:10.206743 (+ 1248us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:38:10.206746 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:38:10.206796 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:38:10.206871 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:38:10.208012 (+ 1141us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:38:10.208544 (+ 532us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:38:10.209801 (+ 1257us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:38:10.210072 (+ 271us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:38:10.210200 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:38:10.213086 (+ 2886us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:38:10.213090 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:38:10.213093 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 18:38:10.213119 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:38:10.214823 (+ 1704us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:38:10.215479 (+ 656us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:38:10.215482 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:38:10.215483 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:38:10.215565 (+ 82us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:38:10.215980 (+ 415us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:38:10.215982 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:38:10.215983 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:38:10.216170 (+ 187us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:38:10.216244 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0806 18:38:10.216926 (+ 682us) server_negotiation.cc:299] Negotiation successful
0806 18:38:10.217036 (+ 110us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260806 18:38:10.218497 5913 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:38:10.204889 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.194.62:43945 (local address 127.19.194.2:46487)
0806 18:38:10.205336 (+ 447us) negotiation.cc:107] Waiting for socket to connect
0806 18:38:10.205375 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0806 18:38:10.206548 (+ 1173us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:38:10.207011 (+ 463us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:38:10.207017 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:38:10.207408 (+ 391us) client_negotiation.cc:189] Negotiated authn=SASL
0806 18:38:10.207876 (+ 468us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:38:10.207889 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:38:10.208652 (+ 763us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:38:10.208654 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:38:10.209668 (+ 1014us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:38:10.209681 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:38:10.210238 (+ 557us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:38:10.210243 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:38:10.210474 (+ 231us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:38:10.211176 (+ 702us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 18:38:10.211202 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0806 18:38:10.212927 (+ 1725us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 18:38:10.214955 (+ 2028us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:38:10.214965 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:38:10.214983 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:38:10.215373 (+ 390us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:38:10.215677 (+ 304us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:38:10.215683 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:38:10.215688 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:38:10.215871 (+ 183us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:38:10.216276 (+ 405us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 18:38:10.216285 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 18:38:10.216618 (+ 333us) client_negotiation.cc:769] Sending connection context
0806 18:38:10.216830 (+ 212us) client_negotiation.cc:240] Negotiation successful
0806 18:38:10.217205 (+ 375us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":258,"thread_start_us":113,"threads_started":1}
I20260806 18:38:10.219777 5911 heartbeater.cc:347] Connected to a master server at 127.19.194.62:43945
I20260806 18:38:10.220016 5911 heartbeater.cc:464] Registering TS with master...
I20260806 18:38:10.220474 5911 heartbeater.cc:511] Master 127.19.194.62:43945 requested a full tablet report, sending...
WARNING: no policy specified for kudu/127.19.194.3@KRBTEST.COM; defaulting to no policy
I20260806 18:38:10.221510 5565 ts_manager.cc:194] Registered new tserver with Master: 10225ccf5e6d4fb884371ff6c8dd4b9a (127.19.194.2:40689)
I20260806 18:38:10.222087 5565 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.19.194.2@KRBTEST.COM'} at 127.19.194.2:46487
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.19.194.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.19.194.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786041246094603-20232-0/krb5kdc/kudu_127.19.194.3.keytab.
Entry for principal kudu/127.19.194.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786041246094603-20232-0/krb5kdc/kudu_127.19.194.3.keytab.
I20260806 18:38:10.240300 20232 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJW69tn/build/debug/bin/kudu
/tmp/dist-test-taskJW69tn/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786041246094603-20232-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786041246094603-20232-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786041246094603-20232-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786041246094603-20232-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.19.194.3:0
--local_ip_for_outbound_sockets=127.19.194.3
--webserver_interface=127.19.194.3
--webserver_port=0
--tserver_master_addrs=127.19.194.62:43945
--keytab_file=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786041246094603-20232-0/krb5kdc/kudu_127.19.194.3.keytab
--principal=kudu/127.19.194.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.19.194.20:36137
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786041246094603-20232-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786041246094603-20232-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786041246094603-20232-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 18:38:10.350355 5916 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 18:38:10.350586 5916 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 18:38:10.350627 5916 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 18:38:10.354473 5916 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 18:38:10.354521 5916 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 18:38:10.354593 5916 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.19.194.3
I20260806 18:38:10.359916 5916 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.194.20:36137
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786041246094603-20232-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786041246094603-20232-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786041246094603-20232-0/krb5kdc/kudu_127.19.194.3.keytab
--principal=kudu/127.19.194.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.19.194.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786041246094603-20232-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.19.194.3
--webserver_port=0
--tserver_master_addrs=127.19.194.62:43945
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5916
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.19.194.3
--log_dir=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786041246094603-20232-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 17:59:13 UTC on ccycloud.quasar-wvsnzx.root.comops.site
build id 4571028
I20260806 18:38:10.360862 5916 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 18:38:10.361789 5916 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 18:38:10.371153 5922 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:38:10.371095 5923 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:38:10.371086 5925 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 18:38:10.371515 5916 server_base.cc:1034] running on GCE node
I20260806 18:38:10.371843 5916 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 18:38:10.372440 5916 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 18:38:10.373596 5916 hybrid_clock.cc:648] HybridClock initialized: now 1786041490373578 us; error 30 us; skew 500 ppm
Aug 06 18:38:10 dist-test-slave-1xrh krb5kdc[5517](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041490, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.194.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 18:38:10.376395 5916 init.cc:377] Logged in from keytab as kudu/127.19.194.3@KRBTEST.COM (short username kudu)
I20260806 18:38:10.377128 5916 webserver.cc:460] Webserver started at http://127.19.194.3:36849/ using document root <none> and password file <none>
I20260806 18:38:10.377599 5916 fs_manager.cc:359] Metadata directory not provided
I20260806 18:38:10.377652 5916 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 18:38:10.377800 5916 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 18:38:10.379709 5916 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786041246094603-20232-0/minicluster-data/ts-2/data/instance:
uuid: "e432285bc78f410eb2258d2a7098dd4f"
format_stamp: "Formatted at 2026-08-06 18:38:10 on dist-test-slave-1xrh"
server_key: "85c120d2f4a2fc669798418455fc7859"
server_key_iv: "534be26706858312c7ababe3d1a904f7"
server_key_version: "encryptionkey@0"
I20260806 18:38:10.380306 5916 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786041246094603-20232-0/minicluster-data/ts-2/wal/instance:
uuid: "e432285bc78f410eb2258d2a7098dd4f"
format_stamp: "Formatted at 2026-08-06 18:38:10 on dist-test-slave-1xrh"
server_key: "85c120d2f4a2fc669798418455fc7859"
server_key_iv: "534be26706858312c7ababe3d1a904f7"
server_key_version: "encryptionkey@0"
I20260806 18:38:10.384056 5916 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260806 18:38:10.386297 5932 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 18:38:10.387511 5916 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260806 18:38:10.387604 5916 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786041246094603-20232-0/minicluster-data/ts-2/data,/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786041246094603-20232-0/minicluster-data/ts-2/wal
uuid: "e432285bc78f410eb2258d2a7098dd4f"
format_stamp: "Formatted at 2026-08-06 18:38:10 on dist-test-slave-1xrh"
server_key: "85c120d2f4a2fc669798418455fc7859"
server_key_iv: "534be26706858312c7ababe3d1a904f7"
server_key_version: "encryptionkey@0"
I20260806 18:38:10.387682 5916 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786041246094603-20232-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786041246094603-20232-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786041246094603-20232-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 18:38:10.425266 5916 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 18:38:10.428661 5916 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 18:38:10.428838 5916 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 18:38:10.429371 5916 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 18:38:10.430143 5916 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 18:38:10.430191 5916 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 18:38:10.430230 5916 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 18:38:10.430250 5916 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 18:38:10.439061 5916 rpc_server.cc:307] RPC server started. Bound to: 127.19.194.3:41605
I20260806 18:38:10.439091 6055 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.194.3:41605 every 8 connection(s)
I20260806 18:38:10.439976 5916 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786041246094603-20232-0/minicluster-data/ts-2/data/info.pb
I20260806 18:38:10.446547 20232 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJW69tn/build/debug/bin/kudu as pid 5916
I20260806 18:38:10.446702 20232 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786041246094603-20232-0/minicluster-data/ts-2/wal/instance
I20260806 18:38:10.447077 20232 external_mini_cluster.cc:1428] Setting key afeb0af8de88d64cbdb26bae7fd65273
Aug 06 18:38:10 dist-test-slave-1xrh krb5kdc[5517](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041490, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.194.3@KRBTEST.COM for kudu/127.19.194.62@KRBTEST.COM
I20260806 18:38:10.453526 5608 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:38:10.441795 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.194.3:40581 (local address 127.19.194.62:43945)
0806 18:38:10.441974 (+ 179us) server_negotiation.cc:206] Beginning negotiation
0806 18:38:10.441979 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 18:38:10.442828 (+ 849us) server_negotiation.cc:407] Connection header received
0806 18:38:10.444159 (+ 1331us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:38:10.444162 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:38:10.444212 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:38:10.444290 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:38:10.445462 (+ 1172us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:38:10.446011 (+ 549us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:38:10.447346 (+ 1335us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:38:10.447611 (+ 265us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:38:10.447700 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:38:10.450528 (+ 2828us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:38:10.450531 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:38:10.450533 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 18:38:10.450556 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:38:10.451900 (+ 1344us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:38:10.452360 (+ 460us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:38:10.452363 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:38:10.452364 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:38:10.452432 (+ 68us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:38:10.452705 (+ 273us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:38:10.452707 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:38:10.452708 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:38:10.452857 (+ 149us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:38:10.452938 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0806 18:38:10.453324 (+ 386us) server_negotiation.cc:299] Negotiation successful
0806 18:38:10.453413 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":80}
I20260806 18:38:10.454290 6058 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:38:10.442236 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.194.62:43945 (local address 127.19.194.3:40581)
0806 18:38:10.442672 (+ 436us) negotiation.cc:107] Waiting for socket to connect
0806 18:38:10.442713 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0806 18:38:10.443964 (+ 1251us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:38:10.444449 (+ 485us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:38:10.444454 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:38:10.444777 (+ 323us) client_negotiation.cc:189] Negotiated authn=SASL
0806 18:38:10.445240 (+ 463us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:38:10.445253 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:38:10.446131 (+ 878us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:38:10.446134 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:38:10.447213 (+ 1079us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:38:10.447225 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:38:10.447763 (+ 538us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:38:10.447768 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:38:10.448008 (+ 240us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:38:10.448702 (+ 694us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 18:38:10.448721 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0806 18:38:10.450370 (+ 1649us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 18:38:10.452013 (+ 1643us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:38:10.452017 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:38:10.452028 (+ 11us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:38:10.452278 (+ 250us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:38:10.452507 (+ 229us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:38:10.452509 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:38:10.452511 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:38:10.452631 (+ 120us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:38:10.452939 (+ 308us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 18:38:10.452943 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 18:38:10.453137 (+ 194us) client_negotiation.cc:769] Sending connection context
0806 18:38:10.453278 (+ 141us) client_negotiation.cc:240] Negotiation successful
0806 18:38:10.453534 (+ 256us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":263,"thread_start_us":116,"threads_started":1}
I20260806 18:38:10.455350 6056 heartbeater.cc:347] Connected to a master server at 127.19.194.62:43945
I20260806 18:38:10.455585 6056 heartbeater.cc:464] Registering TS with master...
I20260806 18:38:10.456071 6056 heartbeater.cc:511] Master 127.19.194.62:43945 requested a full tablet report, sending...
I20260806 18:38:10.457037 5565 ts_manager.cc:194] Registered new tserver with Master: e432285bc78f410eb2258d2a7098dd4f (127.19.194.3:41605)
I20260806 18:38:10.457599 5565 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.19.194.3@KRBTEST.COM'} at 127.19.194.3:40581
I20260806 18:38:10.460906 20232 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260806 18:38:10.469507 5608 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:38:10.462750 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51326 (local address 127.19.194.62:43945)
0806 18:38:10.462855 (+ 105us) server_negotiation.cc:206] Beginning negotiation
0806 18:38:10.462859 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 18:38:10.463021 (+ 162us) server_negotiation.cc:407] Connection header received
0806 18:38:10.463099 (+ 78us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:38:10.463101 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:38:10.463169 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:38:10.463245 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:38:10.463902 (+ 657us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:38:10.464357 (+ 455us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:38:10.465115 (+ 758us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:38:10.465325 (+ 210us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:38:10.465407 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:38:10.466556 (+ 1149us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:38:10.466559 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:38:10.466561 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 18:38:10.466582 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:38:10.468075 (+ 1493us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:38:10.468438 (+ 363us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:38:10.468440 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:38:10.468442 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:38:10.468509 (+ 67us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:38:10.468757 (+ 248us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:38:10.468760 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:38:10.468762 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:38:10.469021 (+ 259us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:38:10.469122 (+ 101us) server_negotiation.cc:1035] Waiting for connection context
0806 18:38:10.469312 (+ 190us) server_negotiation.cc:299] Negotiation successful
0806 18:38:10.469394 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260806 18:38:10.472070 5565 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:51326:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260806 18:38:10.474892 5565 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 18:38:10.488458 6067 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:38:10.483176 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.194.3:41605 (local address 127.0.0.1:50140)
0806 18:38:10.483613 (+ 437us) negotiation.cc:107] Waiting for socket to connect
0806 18:38:10.483654 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0806 18:38:10.483771 (+ 117us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:38:10.484323 (+ 552us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:38:10.484331 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:38:10.484354 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 18:38:10.484502 (+ 148us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:38:10.484509 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:38:10.485965 (+ 1456us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:38:10.485970 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:38:10.486792 (+ 822us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:38:10.486801 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:38:10.487969 (+ 1168us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:38:10.487973 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:38:10.488090 (+ 117us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:38:10.488127 (+ 37us) client_negotiation.cc:769] Sending connection context
0806 18:38:10.488202 (+ 75us) client_negotiation.cc:240] Negotiation successful
0806 18:38:10.488318 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":284,"thread_start_us":101,"threads_started":1}
I20260806 18:38:10.489197 6068 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:38:10.483330 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50140 (local address 127.19.194.3:41605)
0806 18:38:10.483808 (+ 478us) server_negotiation.cc:206] Beginning negotiation
0806 18:38:10.483821 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0806 18:38:10.483845 (+ 24us) server_negotiation.cc:407] Connection header received
0806 18:38:10.483929 (+ 84us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:38:10.483935 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:38:10.484138 (+ 203us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:38:10.484652 (+ 514us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 18:38:10.484802 (+ 150us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:38:10.485834 (+ 1032us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:38:10.486928 (+ 1094us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:38:10.487820 (+ 892us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:38:10.488823 (+ 1003us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:38:10.488936 (+ 113us) server_negotiation.cc:1035] Waiting for connection context
0806 18:38:10.488975 (+ 39us) server_negotiation.cc:299] Negotiation successful
0806 18:38:10.489037 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":372,"thread_start_us":161,"threads_started":1}
I20260806 18:38:10.489828 6070 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:38:10.483511 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.194.1:37765 (local address 127.0.0.1:40490)
0806 18:38:10.484141 (+ 630us) negotiation.cc:107] Waiting for socket to connect
0806 18:38:10.484154 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0806 18:38:10.484244 (+ 90us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:38:10.485236 (+ 992us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:38:10.485239 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:38:10.485275 (+ 36us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 18:38:10.485420 (+ 145us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:38:10.485427 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:38:10.487005 (+ 1578us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:38:10.487010 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:38:10.488140 (+ 1130us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:38:10.488150 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:38:10.489200 (+ 1050us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:38:10.489206 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:38:10.489503 (+ 297us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:38:10.489530 (+ 27us) client_negotiation.cc:769] Sending connection context
0806 18:38:10.489606 (+ 76us) client_negotiation.cc:240] Negotiation successful
0806 18:38:10.489675 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":566,"mutex_wait_us":23,"spinlock_wait_cycles":3968,"thread_start_us":76,"threads_started":1}
I20260806 18:38:10.489858 6072 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:38:10.484328 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37082 (local address 127.19.194.2:40689)
0806 18:38:10.484660 (+ 332us) server_negotiation.cc:206] Beginning negotiation
0806 18:38:10.484672 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0806 18:38:10.484694 (+ 22us) server_negotiation.cc:407] Connection header received
0806 18:38:10.484768 (+ 74us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:38:10.484773 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:38:10.484949 (+ 176us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:38:10.485142 (+ 193us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 18:38:10.485591 (+ 449us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:38:10.486776 (+ 1185us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:38:10.488344 (+ 1568us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:38:10.489077 (+ 733us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:38:10.489169 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:38:10.489312 (+ 143us) server_negotiation.cc:1035] Waiting for connection context
0806 18:38:10.489619 (+ 307us) server_negotiation.cc:299] Negotiation successful
0806 18:38:10.489713 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":230,"thread_start_us":131,"threads_started":1}
I20260806 18:38:10.491873 5987 tablet_service.cc:1467] Processing CreateTablet for tablet 3758498403404ed582d5250dab504aff (DEFAULT_TABLE table=test-table [id=dcd56a0cfa5446c28c1350ce1fac1777]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 18:38:10.492856 5987 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3758498403404ed582d5250dab504aff. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 18:38:10.489838 6069 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:38:10.483901 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.194.2:40689 (local address 127.0.0.1:37082)
0806 18:38:10.484394 (+ 493us) negotiation.cc:107] Waiting for socket to connect
0806 18:38:10.484420 (+ 26us) client_negotiation.cc:174] Beginning negotiation
0806 18:38:10.484501 (+ 81us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:38:10.485280 (+ 779us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:38:10.485283 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:38:10.485305 (+ 22us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 18:38:10.485454 (+ 149us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:38:10.485461 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:38:10.486961 (+ 1500us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:38:10.486964 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:38:10.488249 (+ 1285us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:38:10.488257 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:38:10.489386 (+ 1129us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:38:10.489389 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:38:10.489477 (+ 88us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:38:10.489494 (+ 17us) client_negotiation.cc:769] Sending connection context
0806 18:38:10.489586 (+ 92us) client_negotiation.cc:240] Negotiation successful
0806 18:38:10.489661 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":436,"mutex_wait_us":175,"thread_start_us":109,"threads_started":1}
I20260806 18:38:10.495265 6071 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:38:10.483966 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40490 (local address 127.19.194.1:37765)
0806 18:38:10.484770 (+ 804us) server_negotiation.cc:206] Beginning negotiation
0806 18:38:10.484784 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0806 18:38:10.484804 (+ 20us) server_negotiation.cc:407] Connection header received
0806 18:38:10.484868 (+ 64us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:38:10.484872 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:38:10.485072 (+ 200us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:38:10.485578 (+ 506us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 18:38:10.485746 (+ 168us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:38:10.486839 (+ 1093us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:38:10.488263 (+ 1424us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:38:10.489049 (+ 786us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:38:10.489165 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:38:10.489312 (+ 147us) server_negotiation.cc:1035] Waiting for connection context
0806 18:38:10.494973 (+ 5661us) server_negotiation.cc:299] Negotiation successful
0806 18:38:10.495082 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":710,"thread_start_us":117,"threads_started":1}
I20260806 18:38:10.497545 5703 tablet_service.cc:1467] Processing CreateTablet for tablet 3758498403404ed582d5250dab504aff (DEFAULT_TABLE table=test-table [id=dcd56a0cfa5446c28c1350ce1fac1777]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 18:38:10.497730 5842 tablet_service.cc:1467] Processing CreateTablet for tablet 3758498403404ed582d5250dab504aff (DEFAULT_TABLE table=test-table [id=dcd56a0cfa5446c28c1350ce1fac1777]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 18:38:10.498472 5703 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3758498403404ed582d5250dab504aff. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 18:38:10.498730 5842 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3758498403404ed582d5250dab504aff. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 18:38:10.504498 6074 tablet_bootstrap.cc:492] T 3758498403404ed582d5250dab504aff P 8219dd885a9d4d3ab75e45276b98997d: Bootstrap starting.
I20260806 18:38:10.505203 6073 tablet_bootstrap.cc:492] T 3758498403404ed582d5250dab504aff P e432285bc78f410eb2258d2a7098dd4f: Bootstrap starting.
I20260806 18:38:10.505404 6075 tablet_bootstrap.cc:492] T 3758498403404ed582d5250dab504aff P 10225ccf5e6d4fb884371ff6c8dd4b9a: Bootstrap starting.
I20260806 18:38:10.506613 6074 tablet_bootstrap.cc:654] T 3758498403404ed582d5250dab504aff P 8219dd885a9d4d3ab75e45276b98997d: Neither blocks nor log segments found. Creating new log.
I20260806 18:38:10.507329 6073 tablet_bootstrap.cc:654] T 3758498403404ed582d5250dab504aff P e432285bc78f410eb2258d2a7098dd4f: Neither blocks nor log segments found. Creating new log.
I20260806 18:38:10.507822 6075 tablet_bootstrap.cc:654] T 3758498403404ed582d5250dab504aff P 10225ccf5e6d4fb884371ff6c8dd4b9a: Neither blocks nor log segments found. Creating new log.
I20260806 18:38:10.507846 6074 log.cc:824] T 3758498403404ed582d5250dab504aff P 8219dd885a9d4d3ab75e45276b98997d: Log is configured to *not* fsync() on all Append() calls
I20260806 18:38:10.508615 6075 log.cc:824] T 3758498403404ed582d5250dab504aff P 10225ccf5e6d4fb884371ff6c8dd4b9a: Log is configured to *not* fsync() on all Append() calls
I20260806 18:38:10.508615 6073 log.cc:824] T 3758498403404ed582d5250dab504aff P e432285bc78f410eb2258d2a7098dd4f: Log is configured to *not* fsync() on all Append() calls
I20260806 18:38:10.509760 6074 tablet_bootstrap.cc:492] T 3758498403404ed582d5250dab504aff P 8219dd885a9d4d3ab75e45276b98997d: No bootstrap required, opened a new log
I20260806 18:38:10.509960 6074 ts_tablet_manager.cc:1397] T 3758498403404ed582d5250dab504aff P 8219dd885a9d4d3ab75e45276b98997d: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260806 18:38:10.510306 6075 tablet_bootstrap.cc:492] T 3758498403404ed582d5250dab504aff P 10225ccf5e6d4fb884371ff6c8dd4b9a: No bootstrap required, opened a new log
I20260806 18:38:10.510499 6075 ts_tablet_manager.cc:1397] T 3758498403404ed582d5250dab504aff P 10225ccf5e6d4fb884371ff6c8dd4b9a: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260806 18:38:10.511092 6073 tablet_bootstrap.cc:492] T 3758498403404ed582d5250dab504aff P e432285bc78f410eb2258d2a7098dd4f: No bootstrap required, opened a new log
I20260806 18:38:10.511286 6073 ts_tablet_manager.cc:1397] T 3758498403404ed582d5250dab504aff P e432285bc78f410eb2258d2a7098dd4f: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.001s
I20260806 18:38:10.513365 6074 raft_consensus.cc:348] T 3758498403404ed582d5250dab504aff P 8219dd885a9d4d3ab75e45276b98997d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e432285bc78f410eb2258d2a7098dd4f" member_type: VOTER last_known_addr { host: "127.19.194.3" port: 41605 } } peers { permanent_uuid: "10225ccf5e6d4fb884371ff6c8dd4b9a" member_type: VOTER last_known_addr { host: "127.19.194.2" port: 40689 } } peers { permanent_uuid: "8219dd885a9d4d3ab75e45276b98997d" member_type: VOTER last_known_addr { host: "127.19.194.1" port: 37765 } }
I20260806 18:38:10.513602 6074 raft_consensus.cc:374] T 3758498403404ed582d5250dab504aff P 8219dd885a9d4d3ab75e45276b98997d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 18:38:10.513655 6074 raft_consensus.cc:729] T 3758498403404ed582d5250dab504aff P 8219dd885a9d4d3ab75e45276b98997d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8219dd885a9d4d3ab75e45276b98997d, State: Initialized, Role: FOLLOWER
I20260806 18:38:10.514106 6074 consensus_queue.cc:260] T 3758498403404ed582d5250dab504aff P 8219dd885a9d4d3ab75e45276b98997d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e432285bc78f410eb2258d2a7098dd4f" member_type: VOTER last_known_addr { host: "127.19.194.3" port: 41605 } } peers { permanent_uuid: "10225ccf5e6d4fb884371ff6c8dd4b9a" member_type: VOTER last_known_addr { host: "127.19.194.2" port: 40689 } } peers { permanent_uuid: "8219dd885a9d4d3ab75e45276b98997d" member_type: VOTER last_known_addr { host: "127.19.194.1" port: 37765 } }
I20260806 18:38:10.514225 6075 raft_consensus.cc:348] T 3758498403404ed582d5250dab504aff P 10225ccf5e6d4fb884371ff6c8dd4b9a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e432285bc78f410eb2258d2a7098dd4f" member_type: VOTER last_known_addr { host: "127.19.194.3" port: 41605 } } peers { permanent_uuid: "10225ccf5e6d4fb884371ff6c8dd4b9a" member_type: VOTER last_known_addr { host: "127.19.194.2" port: 40689 } } peers { permanent_uuid: "8219dd885a9d4d3ab75e45276b98997d" member_type: VOTER last_known_addr { host: "127.19.194.1" port: 37765 } }
I20260806 18:38:10.514494 6075 raft_consensus.cc:374] T 3758498403404ed582d5250dab504aff P 10225ccf5e6d4fb884371ff6c8dd4b9a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 18:38:10.514561 6075 raft_consensus.cc:729] T 3758498403404ed582d5250dab504aff P 10225ccf5e6d4fb884371ff6c8dd4b9a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 10225ccf5e6d4fb884371ff6c8dd4b9a, State: Initialized, Role: FOLLOWER
I20260806 18:38:10.514837 5771 heartbeater.cc:503] Master 127.19.194.62:43945 was elected leader, sending a full tablet report...
I20260806 18:38:10.515075 6074 ts_tablet_manager.cc:1428] T 3758498403404ed582d5250dab504aff P 8219dd885a9d4d3ab75e45276b98997d: Time spent starting tablet: real 0.005s user 0.006s sys 0.000s
I20260806 18:38:10.514942 6073 raft_consensus.cc:348] T 3758498403404ed582d5250dab504aff P e432285bc78f410eb2258d2a7098dd4f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e432285bc78f410eb2258d2a7098dd4f" member_type: VOTER last_known_addr { host: "127.19.194.3" port: 41605 } } peers { permanent_uuid: "10225ccf5e6d4fb884371ff6c8dd4b9a" member_type: VOTER last_known_addr { host: "127.19.194.2" port: 40689 } } peers { permanent_uuid: "8219dd885a9d4d3ab75e45276b98997d" member_type: VOTER last_known_addr { host: "127.19.194.1" port: 37765 } }
I20260806 18:38:10.515194 6073 raft_consensus.cc:374] T 3758498403404ed582d5250dab504aff P e432285bc78f410eb2258d2a7098dd4f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 18:38:10.515260 6073 raft_consensus.cc:729] T 3758498403404ed582d5250dab504aff P e432285bc78f410eb2258d2a7098dd4f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e432285bc78f410eb2258d2a7098dd4f, State: Initialized, Role: FOLLOWER
I20260806 18:38:10.515648 6073 consensus_queue.cc:260] T 3758498403404ed582d5250dab504aff P e432285bc78f410eb2258d2a7098dd4f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e432285bc78f410eb2258d2a7098dd4f" member_type: VOTER last_known_addr { host: "127.19.194.3" port: 41605 } } peers { permanent_uuid: "10225ccf5e6d4fb884371ff6c8dd4b9a" member_type: VOTER last_known_addr { host: "127.19.194.2" port: 40689 } } peers { permanent_uuid: "8219dd885a9d4d3ab75e45276b98997d" member_type: VOTER last_known_addr { host: "127.19.194.1" port: 37765 } }
I20260806 18:38:10.516086 6075 consensus_queue.cc:260] T 3758498403404ed582d5250dab504aff P 10225ccf5e6d4fb884371ff6c8dd4b9a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e432285bc78f410eb2258d2a7098dd4f" member_type: VOTER last_known_addr { host: "127.19.194.3" port: 41605 } } peers { permanent_uuid: "10225ccf5e6d4fb884371ff6c8dd4b9a" member_type: VOTER last_known_addr { host: "127.19.194.2" port: 40689 } } peers { permanent_uuid: "8219dd885a9d4d3ab75e45276b98997d" member_type: VOTER last_known_addr { host: "127.19.194.1" port: 37765 } }
I20260806 18:38:10.516711 5911 heartbeater.cc:503] Master 127.19.194.62:43945 was elected leader, sending a full tablet report...
I20260806 18:38:10.516953 6075 ts_tablet_manager.cc:1428] T 3758498403404ed582d5250dab504aff P 10225ccf5e6d4fb884371ff6c8dd4b9a: Time spent starting tablet: real 0.006s user 0.005s sys 0.000s
I20260806 18:38:10.516559 6073 ts_tablet_manager.cc:1428] T 3758498403404ed582d5250dab504aff P e432285bc78f410eb2258d2a7098dd4f: Time spent starting tablet: real 0.005s user 0.003s sys 0.002s
I20260806 18:38:10.517760 6056 heartbeater.cc:503] Master 127.19.194.62:43945 was elected leader, sending a full tablet report...
W20260806 18:38:10.691169 6057 tablet.cc:2369] T 3758498403404ed582d5250dab504aff P e432285bc78f410eb2258d2a7098dd4f: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260806 18:38:10.703944 5912 tablet.cc:2369] T 3758498403404ed582d5250dab504aff P 10225ccf5e6d4fb884371ff6c8dd4b9a: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 18:38:10.705456 6080 raft_consensus.cc:482] T 3758498403404ed582d5250dab504aff P e432285bc78f410eb2258d2a7098dd4f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 18:38:10.705662 6080 raft_consensus.cc:504] T 3758498403404ed582d5250dab504aff P e432285bc78f410eb2258d2a7098dd4f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e432285bc78f410eb2258d2a7098dd4f" member_type: VOTER last_known_addr { host: "127.19.194.3" port: 41605 } } peers { permanent_uuid: "10225ccf5e6d4fb884371ff6c8dd4b9a" member_type: VOTER last_known_addr { host: "127.19.194.2" port: 40689 } } peers { permanent_uuid: "8219dd885a9d4d3ab75e45276b98997d" member_type: VOTER last_known_addr { host: "127.19.194.1" port: 37765 } }
I20260806 18:38:10.706797 6080 leader_election.cc:288] T 3758498403404ed582d5250dab504aff P e432285bc78f410eb2258d2a7098dd4f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 10225ccf5e6d4fb884371ff6c8dd4b9a (127.19.194.2:40689), 8219dd885a9d4d3ab75e45276b98997d (127.19.194.1:37765)
I20260806 18:38:10.710731 6058 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:38:10.707082 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.194.2:40689 (local address 127.19.194.3:44237)
0806 18:38:10.707212 (+ 130us) negotiation.cc:107] Waiting for socket to connect
0806 18:38:10.707228 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0806 18:38:10.707334 (+ 106us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:38:10.707657 (+ 323us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:38:10.707660 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:38:10.707684 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 18:38:10.707819 (+ 135us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:38:10.707829 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:38:10.708651 (+ 822us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:38:10.708655 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:38:10.709584 (+ 929us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:38:10.709594 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:38:10.710259 (+ 665us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:38:10.710266 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:38:10.710378 (+ 112us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:38:10.710442 (+ 64us) client_negotiation.cc:769] Sending connection context
0806 18:38:10.710504 (+ 62us) client_negotiation.cc:240] Negotiation successful
0806 18:38:10.710586 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":49,"mutex_wait_us":56}
I20260806 18:38:10.710747 6072 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:38:10.707217 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.194.3:44237 (local address 127.19.194.2:40689)
0806 18:38:10.707344 (+ 127us) server_negotiation.cc:206] Beginning negotiation
0806 18:38:10.707349 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 18:38:10.707367 (+ 18us) server_negotiation.cc:407] Connection header received
0806 18:38:10.707442 (+ 75us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:38:10.707445 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:38:10.707502 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:38:10.707602 (+ 100us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 18:38:10.707947 (+ 345us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:38:10.708535 (+ 588us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:38:10.709726 (+ 1191us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:38:10.710133 (+ 407us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:38:10.710226 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:38:10.710294 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0806 18:38:10.710546 (+ 252us) server_negotiation.cc:299] Negotiation successful
0806 18:38:10.710626 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260806 18:38:10.710822 6071 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:38:10.707226 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.194.3:50441 (local address 127.19.194.1:37765)
0806 18:38:10.707344 (+ 118us) server_negotiation.cc:206] Beginning negotiation
0806 18:38:10.707349 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 18:38:10.707657 (+ 308us) server_negotiation.cc:407] Connection header received
0806 18:38:10.707727 (+ 70us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:38:10.707730 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:38:10.707783 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:38:10.707872 (+ 89us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 18:38:10.708184 (+ 312us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:38:10.708721 (+ 537us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:38:10.709805 (+ 1084us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:38:10.710211 (+ 406us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:38:10.710324 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:38:10.710376 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0806 18:38:10.710624 (+ 248us) server_negotiation.cc:299] Negotiation successful
0806 18:38:10.710700 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260806 18:38:10.710999 6082 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:38:10.707082 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.194.1:37765 (local address 127.19.194.3:50441)
0806 18:38:10.707536 (+ 454us) negotiation.cc:107] Waiting for socket to connect
0806 18:38:10.707552 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0806 18:38:10.707651 (+ 99us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:38:10.707905 (+ 254us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:38:10.707908 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:38:10.707936 (+ 28us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 18:38:10.708066 (+ 130us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:38:10.708072 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:38:10.708833 (+ 761us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:38:10.708836 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:38:10.709648 (+ 812us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:38:10.709679 (+ 31us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:38:10.710341 (+ 662us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:38:10.710344 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:38:10.710467 (+ 123us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:38:10.710505 (+ 38us) client_negotiation.cc:769] Sending connection context
0806 18:38:10.710583 (+ 78us) client_negotiation.cc:240] Negotiation successful
0806 18:38:10.710644 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":379,"mutex_wait_us":183,"thread_start_us":119,"threads_started":1}
I20260806 18:38:10.711539 5864 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3758498403404ed582d5250dab504aff" candidate_uuid: "e432285bc78f410eb2258d2a7098dd4f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "10225ccf5e6d4fb884371ff6c8dd4b9a" is_pre_election: true
I20260806 18:38:10.711644 5723 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3758498403404ed582d5250dab504aff" candidate_uuid: "e432285bc78f410eb2258d2a7098dd4f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8219dd885a9d4d3ab75e45276b98997d" is_pre_election: true
I20260806 18:38:10.711862 5723 raft_consensus.cc:2454] T 3758498403404ed582d5250dab504aff P 8219dd885a9d4d3ab75e45276b98997d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e432285bc78f410eb2258d2a7098dd4f in term 0.
I20260806 18:38:10.711863 5864 raft_consensus.cc:2454] T 3758498403404ed582d5250dab504aff P 10225ccf5e6d4fb884371ff6c8dd4b9a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e432285bc78f410eb2258d2a7098dd4f in term 0.
I20260806 18:38:10.712391 5937 leader_election.cc:302] T 3758498403404ed582d5250dab504aff P e432285bc78f410eb2258d2a7098dd4f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 10225ccf5e6d4fb884371ff6c8dd4b9a, 8219dd885a9d4d3ab75e45276b98997d, e432285bc78f410eb2258d2a7098dd4f; no voters:
I20260806 18:38:10.712673 6080 raft_consensus.cc:2781] T 3758498403404ed582d5250dab504aff P e432285bc78f410eb2258d2a7098dd4f [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 18:38:10.712756 6080 raft_consensus.cc:482] T 3758498403404ed582d5250dab504aff P e432285bc78f410eb2258d2a7098dd4f [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 18:38:10.712800 6080 raft_consensus.cc:3037] T 3758498403404ed582d5250dab504aff P e432285bc78f410eb2258d2a7098dd4f [term 0 FOLLOWER]: Advancing to term 1
I20260806 18:38:10.713891 6080 raft_consensus.cc:504] T 3758498403404ed582d5250dab504aff P e432285bc78f410eb2258d2a7098dd4f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e432285bc78f410eb2258d2a7098dd4f" member_type: VOTER last_known_addr { host: "127.19.194.3" port: 41605 } } peers { permanent_uuid: "10225ccf5e6d4fb884371ff6c8dd4b9a" member_type: VOTER last_known_addr { host: "127.19.194.2" port: 40689 } } peers { permanent_uuid: "8219dd885a9d4d3ab75e45276b98997d" member_type: VOTER last_known_addr { host: "127.19.194.1" port: 37765 } }
I20260806 18:38:10.714259 6080 leader_election.cc:288] T 3758498403404ed582d5250dab504aff P e432285bc78f410eb2258d2a7098dd4f [CANDIDATE]: Term 1 election: Requested vote from peers 10225ccf5e6d4fb884371ff6c8dd4b9a (127.19.194.2:40689), 8219dd885a9d4d3ab75e45276b98997d (127.19.194.1:37765)
I20260806 18:38:10.714619 5723 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3758498403404ed582d5250dab504aff" candidate_uuid: "e432285bc78f410eb2258d2a7098dd4f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8219dd885a9d4d3ab75e45276b98997d"
I20260806 18:38:10.714749 5864 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3758498403404ed582d5250dab504aff" candidate_uuid: "e432285bc78f410eb2258d2a7098dd4f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "10225ccf5e6d4fb884371ff6c8dd4b9a"
I20260806 18:38:10.714740 5723 raft_consensus.cc:3037] T 3758498403404ed582d5250dab504aff P 8219dd885a9d4d3ab75e45276b98997d [term 0 FOLLOWER]: Advancing to term 1
I20260806 18:38:10.714874 5864 raft_consensus.cc:3037] T 3758498403404ed582d5250dab504aff P 10225ccf5e6d4fb884371ff6c8dd4b9a [term 0 FOLLOWER]: Advancing to term 1
I20260806 18:38:10.715901 5723 raft_consensus.cc:2454] T 3758498403404ed582d5250dab504aff P 8219dd885a9d4d3ab75e45276b98997d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e432285bc78f410eb2258d2a7098dd4f in term 1.
I20260806 18:38:10.716042 5864 raft_consensus.cc:2454] T 3758498403404ed582d5250dab504aff P 10225ccf5e6d4fb884371ff6c8dd4b9a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e432285bc78f410eb2258d2a7098dd4f in term 1.
I20260806 18:38:10.716226 5937 leader_election.cc:302] T 3758498403404ed582d5250dab504aff P e432285bc78f410eb2258d2a7098dd4f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8219dd885a9d4d3ab75e45276b98997d, e432285bc78f410eb2258d2a7098dd4f; no voters:
I20260806 18:38:10.716439 6080 raft_consensus.cc:2781] T 3758498403404ed582d5250dab504aff P e432285bc78f410eb2258d2a7098dd4f [term 1 FOLLOWER]: Leader election won for term 1
I20260806 18:38:10.716635 6080 raft_consensus.cc:686] T 3758498403404ed582d5250dab504aff P e432285bc78f410eb2258d2a7098dd4f [term 1 LEADER]: Becoming Leader. State: Replica: e432285bc78f410eb2258d2a7098dd4f, State: Running, Role: LEADER
I20260806 18:38:10.716951 6080 consensus_queue.cc:237] T 3758498403404ed582d5250dab504aff P e432285bc78f410eb2258d2a7098dd4f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e432285bc78f410eb2258d2a7098dd4f" member_type: VOTER last_known_addr { host: "127.19.194.3" port: 41605 } } peers { permanent_uuid: "10225ccf5e6d4fb884371ff6c8dd4b9a" member_type: VOTER last_known_addr { host: "127.19.194.2" port: 40689 } } peers { permanent_uuid: "8219dd885a9d4d3ab75e45276b98997d" member_type: VOTER last_known_addr { host: "127.19.194.1" port: 37765 } }
I20260806 18:38:10.720005 5565 catalog_manager.cc:5495] T 3758498403404ed582d5250dab504aff P e432285bc78f410eb2258d2a7098dd4f reported cstate change: term changed from 0 to 1, leader changed from <none> to e432285bc78f410eb2258d2a7098dd4f (127.19.194.3). New cstate: current_term: 1 leader_uuid: "e432285bc78f410eb2258d2a7098dd4f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e432285bc78f410eb2258d2a7098dd4f" member_type: VOTER last_known_addr { host: "127.19.194.3" port: 41605 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "10225ccf5e6d4fb884371ff6c8dd4b9a" member_type: VOTER last_known_addr { host: "127.19.194.2" port: 40689 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8219dd885a9d4d3ab75e45276b98997d" member_type: VOTER last_known_addr { host: "127.19.194.1" port: 37765 } health_report { overall_health: UNKNOWN } } }
W20260806 18:38:10.747504 5772 tablet.cc:2369] T 3758498403404ed582d5250dab504aff P 8219dd885a9d4d3ab75e45276b98997d: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 18:38:10.778613 6068 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:38:10.775394 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50154 (local address 127.19.194.3:41605)
0806 18:38:10.775524 (+ 130us) server_negotiation.cc:206] Beginning negotiation
0806 18:38:10.775527 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 18:38:10.775541 (+ 14us) server_negotiation.cc:407] Connection header received
0806 18:38:10.775631 (+ 90us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:38:10.775633 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:38:10.775685 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:38:10.775769 (+ 84us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 18:38:10.776005 (+ 236us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:38:10.776452 (+ 447us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:38:10.777324 (+ 872us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:38:10.777522 (+ 198us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:38:10.777599 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:38:10.777875 (+ 276us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 18:38:10.778281 (+ 406us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 18:38:10.778369 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0806 18:38:10.778446 (+ 77us) server_negotiation.cc:299] Negotiation successful
0806 18:38:10.778492 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260806 18:38:10.784368 5864 raft_consensus.cc:1261] T 3758498403404ed582d5250dab504aff P 10225ccf5e6d4fb884371ff6c8dd4b9a [term 1 FOLLOWER]: Refusing update from remote peer e432285bc78f410eb2258d2a7098dd4f: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 18:38:10.784368 5723 raft_consensus.cc:1261] T 3758498403404ed582d5250dab504aff P 8219dd885a9d4d3ab75e45276b98997d [term 1 FOLLOWER]: Refusing update from remote peer e432285bc78f410eb2258d2a7098dd4f: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 18:38:10.785040 6080 consensus_queue.cc:1035] T 3758498403404ed582d5250dab504aff P e432285bc78f410eb2258d2a7098dd4f [LEADER]: Connected to new peer: Peer: permanent_uuid: "10225ccf5e6d4fb884371ff6c8dd4b9a" member_type: VOTER last_known_addr { host: "127.19.194.2" port: 40689 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 18:38:10.785243 6083 consensus_queue.cc:1035] T 3758498403404ed582d5250dab504aff P e432285bc78f410eb2258d2a7098dd4f [LEADER]: Connected to new peer: Peer: permanent_uuid: "8219dd885a9d4d3ab75e45276b98997d" member_type: VOTER last_known_addr { host: "127.19.194.1" port: 37765 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 18:38:10.793416 6086 mvcc.cc:205] Tried to move back new op lower bound from 7315625946245701632 to 7315625945978953728. Current Snapshot: MvccSnapshot[applied={T|T < 7315625946245701632}]
I20260806 18:38:10.802973 5565 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:51326:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 18:38:10.803172 5565 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:51326:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 18:38:10.805513 5565 catalog_manager.cc:5782] T 00000000000000000000000000000000 P d0be563f6e7e457a93df7a191b190f66: Sending DeleteTablet for 3 replicas of tablet 3758498403404ed582d5250dab504aff
I20260806 18:38:10.806315 5842 tablet_service.cc:1514] Processing DeleteTablet for tablet 3758498403404ed582d5250dab504aff with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 18:38:10 UTC) from {username='kudu', principal='kudu/127.19.194.62@KRBTEST.COM'} at 127.0.0.1:37082
I20260806 18:38:10.806334 5703 tablet_service.cc:1514] Processing DeleteTablet for tablet 3758498403404ed582d5250dab504aff with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 18:38:10 UTC) from {username='kudu', principal='kudu/127.19.194.62@KRBTEST.COM'} at 127.0.0.1:40490
I20260806 18:38:10.806476 5987 tablet_service.cc:1514] Processing DeleteTablet for tablet 3758498403404ed582d5250dab504aff with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 18:38:10 UTC) from {username='kudu', principal='kudu/127.19.194.62@KRBTEST.COM'} at 127.0.0.1:50140
I20260806 18:38:10.806838 6095 tablet_replica.cc:330] T 3758498403404ed582d5250dab504aff P 8219dd885a9d4d3ab75e45276b98997d: stopping tablet replica
I20260806 18:38:10.806887 6097 tablet_replica.cc:330] T 3758498403404ed582d5250dab504aff P e432285bc78f410eb2258d2a7098dd4f: stopping tablet replica
I20260806 18:38:10.807196 6095 raft_consensus.cc:2229] T 3758498403404ed582d5250dab504aff P 8219dd885a9d4d3ab75e45276b98997d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 18:38:10.807196 6097 raft_consensus.cc:2229] T 3758498403404ed582d5250dab504aff P e432285bc78f410eb2258d2a7098dd4f [term 1 LEADER]: Raft consensus shutting down.
I20260806 18:38:10.807482 6095 raft_consensus.cc:2258] T 3758498403404ed582d5250dab504aff P 8219dd885a9d4d3ab75e45276b98997d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 18:38:10.807546 6097 raft_consensus.cc:2258] T 3758498403404ed582d5250dab504aff P e432285bc78f410eb2258d2a7098dd4f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 18:38:10.807847 6096 tablet_replica.cc:330] T 3758498403404ed582d5250dab504aff P 10225ccf5e6d4fb884371ff6c8dd4b9a: stopping tablet replica
I20260806 18:38:10.808117 6096 raft_consensus.cc:2229] T 3758498403404ed582d5250dab504aff P 10225ccf5e6d4fb884371ff6c8dd4b9a [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 18:38:10.808429 6096 raft_consensus.cc:2258] T 3758498403404ed582d5250dab504aff P 10225ccf5e6d4fb884371ff6c8dd4b9a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 18:38:10.808452 20232 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJW69tn/build/debug/bin/kudu with pid 5631
I20260806 18:38:10.809516 6097 ts_tablet_manager.cc:1905] T 3758498403404ed582d5250dab504aff P e432285bc78f410eb2258d2a7098dd4f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 18:38:10.811830 6097 ts_tablet_manager.cc:1918] T 3758498403404ed582d5250dab504aff P e432285bc78f410eb2258d2a7098dd4f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 18:38:10.811908 6097 log.cc:1196] T 3758498403404ed582d5250dab504aff P e432285bc78f410eb2258d2a7098dd4f: Deleting WAL directory at /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786041246094603-20232-0/minicluster-data/ts-2/wal/wals/3758498403404ed582d5250dab504aff
I20260806 18:38:10.812151 6097 ts_tablet_manager.cc:1939] T 3758498403404ed582d5250dab504aff P e432285bc78f410eb2258d2a7098dd4f: Deleting consensus metadata
I20260806 18:38:10.813313 5549 catalog_manager.cc:4841] TS e432285bc78f410eb2258d2a7098dd4f (127.19.194.3:41605): tablet 3758498403404ed582d5250dab504aff (table test-table [id=dcd56a0cfa5446c28c1350ce1fac1777]) successfully deleted
I20260806 18:38:10.813651 6096 ts_tablet_manager.cc:1905] T 3758498403404ed582d5250dab504aff P 10225ccf5e6d4fb884371ff6c8dd4b9a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 18:38:10.815845 6096 ts_tablet_manager.cc:1918] T 3758498403404ed582d5250dab504aff P 10225ccf5e6d4fb884371ff6c8dd4b9a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 18:38:10.815968 6096 log.cc:1196] T 3758498403404ed582d5250dab504aff P 10225ccf5e6d4fb884371ff6c8dd4b9a: Deleting WAL directory at /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786041246094603-20232-0/minicluster-data/ts-1/wal/wals/3758498403404ed582d5250dab504aff
I20260806 18:38:10.816248 6096 ts_tablet_manager.cc:1939] T 3758498403404ed582d5250dab504aff P 10225ccf5e6d4fb884371ff6c8dd4b9a: Deleting consensus metadata
I20260806 18:38:10.817198 5549 catalog_manager.cc:4841] TS 10225ccf5e6d4fb884371ff6c8dd4b9a (127.19.194.2:40689): tablet 3758498403404ed582d5250dab504aff (table test-table [id=dcd56a0cfa5446c28c1350ce1fac1777]) successfully deleted
W20260806 18:38:10.819083 5551 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.19.194.1:37765 (error 108)
I20260806 18:38:10.819909 20232 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJW69tn/build/debug/bin/kudu with pid 5776
I20260806 18:38:10.820109 6069 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:38:10.819777 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.194.1:37765 (local address 127.0.0.1:40492)
0806 18:38:10.819927 (+ 150us) negotiation.cc:107] Waiting for socket to connect
0806 18:38:10.820014 (+ 87us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.194.1:37765: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":54}
W20260806 18:38:10.820261 5551 catalog_manager.cc:4584] TS 8219dd885a9d4d3ab75e45276b98997d (127.19.194.1:37765): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 3758498403404ed582d5250dab504aff: Network error: Client connection negotiation failed: client connection to 127.19.194.1:37765: connect: Connection refused (error 111)
I20260806 18:38:10.833525 20232 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJW69tn/build/debug/bin/kudu with pid 5916
I20260806 18:38:10.840374 20232 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJW69tn/build/debug/bin/kudu with pid 5530
2026-08-06T18:38:10Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/0 (2482 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/1
2026-08-06T18:38:10Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T18:38:10Z Disabled control of system clock
I20260806 18:38:10.871848 20232 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJW69tn/build/debug/bin/kudu
/tmp/dist-test-taskJW69tn/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786041246094603-20232-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786041246094603-20232-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786041246094603-20232-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786041246094603-20232-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.19.194.62:44161
--webserver_interface=127.19.194.62
--webserver_port=0
--builtin_ntp_servers=127.19.194.20:39149
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.19.194.62:44161
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260806 18:38:10.982692 6104 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 18:38:10.982959 6104 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 18:38:10.983026 6104 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 18:38:10.988502 6104 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 18:38:10.988564 6104 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 18:38:10.988579 6104 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 18:38:10.988596 6104 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 18:38:10.988649 6104 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 18:38:10.993412 6104 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.194.20:39149
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786041246094603-20232-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786041246094603-20232-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.19.194.62:44161
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.19.194.62:44161
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786041246094603-20232-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.19.194.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6104
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786041246094603-20232-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 17:59:13 UTC on ccycloud.quasar-wvsnzx.root.comops.site
build id 4571028
I20260806 18:38:10.994514 6104 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 18:38:10.995960 6104 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 18:38:11.004377 6111 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:38:11.004392 6110 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 18:38:11.004489 6104 server_base.cc:1034] running on GCE node
W20260806 18:38:11.004367 6113 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 18:38:11.005160 6104 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 18:38:11.006220 6104 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 18:38:11.007407 6104 hybrid_clock.cc:648] HybridClock initialized: now 1786041491007383 us; error 30 us; skew 500 ppm
I20260806 18:38:11.009471 6104 webserver.cc:460] Webserver started at http://127.19.194.62:45217/ using document root <none> and password file <none>
I20260806 18:38:11.010035 6104 fs_manager.cc:359] Metadata directory not provided
I20260806 18:38:11.010128 6104 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 18:38:11.010351 6104 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 18:38:11.012411 6104 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786041246094603-20232-0/minicluster-data/master-0/data/instance:
uuid: "14d8bae9d1964184bca110daa4f57d1b"
format_stamp: "Formatted at 2026-08-06 18:38:11 on dist-test-slave-1xrh"
server_key: "de2268aff73e09b35575bc743485fb26"
server_key_iv: "aec7fc7081504e6c219282b97a1eef8b"
server_key_version: "encryptionkey@0"
I20260806 18:38:11.013000 6104 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786041246094603-20232-0/minicluster-data/master-0/wal/instance:
uuid: "14d8bae9d1964184bca110daa4f57d1b"
format_stamp: "Formatted at 2026-08-06 18:38:11 on dist-test-slave-1xrh"
server_key: "de2268aff73e09b35575bc743485fb26"
server_key_iv: "aec7fc7081504e6c219282b97a1eef8b"
server_key_version: "encryptionkey@0"
I20260806 18:38:11.016595 6104 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20260806 18:38:11.019022 6119 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 18:38:11.020223 6104 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260806 18:38:11.020366 6104 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786041246094603-20232-0/minicluster-data/master-0/data,/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786041246094603-20232-0/minicluster-data/master-0/wal
uuid: "14d8bae9d1964184bca110daa4f57d1b"
format_stamp: "Formatted at 2026-08-06 18:38:11 on dist-test-slave-1xrh"
server_key: "de2268aff73e09b35575bc743485fb26"
server_key_iv: "aec7fc7081504e6c219282b97a1eef8b"
server_key_version: "encryptionkey@0"
I20260806 18:38:11.020488 6104 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786041246094603-20232-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786041246094603-20232-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786041246094603-20232-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 18:38:11.038101 6104 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 18:38:11.038744 6104 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 18:38:11.038937 6104 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 18:38:11.045977 6104 rpc_server.cc:307] RPC server started. Bound to: 127.19.194.62:44161
I20260806 18:38:11.045989 6177 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.194.62:44161 every 8 connection(s)
I20260806 18:38:11.046875 6104 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786041246094603-20232-0/minicluster-data/master-0/data/info.pb
I20260806 18:38:11.047618 20232 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJW69tn/build/debug/bin/kudu as pid 6104
I20260806 18:38:11.047784 20232 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786041246094603-20232-0/minicluster-data/master-0/wal/instance
I20260806 18:38:11.048101 20232 external_mini_cluster.cc:1428] Setting key f4084285dd1423997f5f965e1eafd10c
I20260806 18:38:11.050508 6178 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 18:38:11.055815 6178 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 14d8bae9d1964184bca110daa4f57d1b: Bootstrap starting.
I20260806 18:38:11.057199 6181 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:38:11.049493 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35726 (local address 127.19.194.62:44161)
0806 18:38:11.050124 (+ 631us) server_negotiation.cc:206] Beginning negotiation
0806 18:38:11.050139 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0806 18:38:11.050173 (+ 34us) server_negotiation.cc:407] Connection header received
0806 18:38:11.050882 (+ 709us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:38:11.050900 (+ 18us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:38:11.051343 (+ 443us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:38:11.051670 (+ 327us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:38:11.052077 (+ 407us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:38:11.053014 (+ 937us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:38:11.053911 (+ 897us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:38:11.054408 (+ 497us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:38:11.054592 (+ 184us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:38:11.055512 (+ 920us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:38:11.055517 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:38:11.055528 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 18:38:11.055921 (+ 393us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:38:11.055985 (+ 64us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 18:38:11.056006 (+ 21us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:38:11.056109 (+ 103us) server_negotiation.cc:1035] Waiting for connection context
0806 18:38:11.056282 (+ 173us) server_negotiation.cc:299] Negotiation successful
0806 18:38:11.056535 (+ 253us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":304,"thread_start_us":150,"threads_started":1}
I20260806 18:38:11.058038 6178 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 14d8bae9d1964184bca110daa4f57d1b: Neither blocks nor log segments found. Creating new log.
I20260806 18:38:11.058774 6178 log.cc:824] T 00000000000000000000000000000000 P 14d8bae9d1964184bca110daa4f57d1b: Log is configured to *not* fsync() on all Append() calls
I20260806 18:38:11.060601 6178 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 14d8bae9d1964184bca110daa4f57d1b: No bootstrap required, opened a new log
I20260806 18:38:11.064144 6178 raft_consensus.cc:348] T 00000000000000000000000000000000 P 14d8bae9d1964184bca110daa4f57d1b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "14d8bae9d1964184bca110daa4f57d1b" member_type: VOTER last_known_addr { host: "127.19.194.62" port: 44161 } }
I20260806 18:38:11.064368 6178 raft_consensus.cc:374] T 00000000000000000000000000000000 P 14d8bae9d1964184bca110daa4f57d1b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 18:38:11.064456 6178 raft_consensus.cc:729] T 00000000000000000000000000000000 P 14d8bae9d1964184bca110daa4f57d1b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 14d8bae9d1964184bca110daa4f57d1b, State: Initialized, Role: FOLLOWER
I20260806 18:38:11.064850 6178 consensus_queue.cc:260] T 00000000000000000000000000000000 P 14d8bae9d1964184bca110daa4f57d1b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "14d8bae9d1964184bca110daa4f57d1b" member_type: VOTER last_known_addr { host: "127.19.194.62" port: 44161 } }
I20260806 18:38:11.065004 6178 raft_consensus.cc:388] T 00000000000000000000000000000000 P 14d8bae9d1964184bca110daa4f57d1b [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 18:38:11.065075 6178 raft_consensus.cc:482] T 00000000000000000000000000000000 P 14d8bae9d1964184bca110daa4f57d1b [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 18:38:11.065212 6178 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 14d8bae9d1964184bca110daa4f57d1b [term 0 FOLLOWER]: Advancing to term 1
I20260806 18:38:11.066026 6178 raft_consensus.cc:504] T 00000000000000000000000000000000 P 14d8bae9d1964184bca110daa4f57d1b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "14d8bae9d1964184bca110daa4f57d1b" member_type: VOTER last_known_addr { host: "127.19.194.62" port: 44161 } }
I20260806 18:38:11.066340 6178 leader_election.cc:302] T 00000000000000000000000000000000 P 14d8bae9d1964184bca110daa4f57d1b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 14d8bae9d1964184bca110daa4f57d1b; no voters:
I20260806 18:38:11.066617 6178 leader_election.cc:288] T 00000000000000000000000000000000 P 14d8bae9d1964184bca110daa4f57d1b [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 18:38:11.066728 6183 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 14d8bae9d1964184bca110daa4f57d1b [term 1 FOLLOWER]: Leader election won for term 1
I20260806 18:38:11.066969 6183 raft_consensus.cc:686] T 00000000000000000000000000000000 P 14d8bae9d1964184bca110daa4f57d1b [term 1 LEADER]: Becoming Leader. State: Replica: 14d8bae9d1964184bca110daa4f57d1b, State: Running, Role: LEADER
I20260806 18:38:11.067296 6183 consensus_queue.cc:237] T 00000000000000000000000000000000 P 14d8bae9d1964184bca110daa4f57d1b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "14d8bae9d1964184bca110daa4f57d1b" member_type: VOTER last_known_addr { host: "127.19.194.62" port: 44161 } }
I20260806 18:38:11.067742 6178 sys_catalog.cc:564] T 00000000000000000000000000000000 P 14d8bae9d1964184bca110daa4f57d1b [sys.catalog]: configured and running, proceeding with master startup.
I20260806 18:38:11.068876 6185 sys_catalog.cc:455] T 00000000000000000000000000000000 P 14d8bae9d1964184bca110daa4f57d1b [sys.catalog]: SysCatalogTable state changed. Reason: New leader 14d8bae9d1964184bca110daa4f57d1b. Latest consensus state: current_term: 1 leader_uuid: "14d8bae9d1964184bca110daa4f57d1b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "14d8bae9d1964184bca110daa4f57d1b" member_type: VOTER last_known_addr { host: "127.19.194.62" port: 44161 } } }
I20260806 18:38:11.068876 6184 sys_catalog.cc:455] T 00000000000000000000000000000000 P 14d8bae9d1964184bca110daa4f57d1b [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "14d8bae9d1964184bca110daa4f57d1b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "14d8bae9d1964184bca110daa4f57d1b" member_type: VOTER last_known_addr { host: "127.19.194.62" port: 44161 } } }
I20260806 18:38:11.069023 6184 sys_catalog.cc:458] T 00000000000000000000000000000000 P 14d8bae9d1964184bca110daa4f57d1b [sys.catalog]: This master's current role is: LEADER
I20260806 18:38:11.069023 6185 sys_catalog.cc:458] T 00000000000000000000000000000000 P 14d8bae9d1964184bca110daa4f57d1b [sys.catalog]: This master's current role is: LEADER
I20260806 18:38:11.069520 6192 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 18:38:11.072465 6192 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 18:38:11.077617 6192 catalog_manager.cc:1347] Generated new cluster ID: 6574a1f4e5fe45fd9d4d2693fdd87535
I20260806 18:38:11.077684 6192 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 18:38:11.086410 6192 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 18:38:11.087258 6192 catalog_manager.cc:1504] Loading token signing keys...
I20260806 18:38:11.093623 6192 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 14d8bae9d1964184bca110daa4f57d1b: Generated new TSK 0
I20260806 18:38:11.094204 6192 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 18:38:11.105342 20232 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJW69tn/build/debug/bin/kudu
/tmp/dist-test-taskJW69tn/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786041246094603-20232-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786041246094603-20232-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786041246094603-20232-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786041246094603-20232-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.19.194.1:0
--local_ip_for_outbound_sockets=127.19.194.1
--webserver_interface=127.19.194.1
--webserver_port=0
--tserver_master_addrs=127.19.194.62:44161
--builtin_ntp_servers=127.19.194.20:39149
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260806 18:38:11.225935 6202 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 18:38:11.226207 6202 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 18:38:11.226282 6202 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 18:38:11.230087 6202 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 18:38:11.230152 6202 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 18:38:11.230248 6202 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.19.194.1
I20260806 18:38:11.235013 6202 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.194.20:39149
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786041246094603-20232-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786041246094603-20232-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.19.194.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786041246094603-20232-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.19.194.1
--webserver_port=0
--tserver_master_addrs=127.19.194.62:44161
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6202
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.19.194.1
--log_dir=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786041246094603-20232-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 17:59:13 UTC on ccycloud.quasar-wvsnzx.root.comops.site
build id 4571028
I20260806 18:38:11.236143 6202 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 18:38:11.237488 6202 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 18:38:11.247063 6208 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:38:11.247056 6210 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:38:11.247056 6207 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 18:38:11.247646 6202 server_base.cc:1034] running on GCE node
I20260806 18:38:11.248008 6202 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 18:38:11.248637 6202 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 18:38:11.249833 6202 hybrid_clock.cc:648] HybridClock initialized: now 1786041491249814 us; error 29 us; skew 500 ppm
I20260806 18:38:11.251603 6202 webserver.cc:460] Webserver started at http://127.19.194.1:36375/ using document root <none> and password file <none>
I20260806 18:38:11.252167 6202 fs_manager.cc:359] Metadata directory not provided
I20260806 18:38:11.252255 6202 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 18:38:11.252480 6202 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 18:38:11.254344 6202 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786041246094603-20232-0/minicluster-data/ts-0/data/instance:
uuid: "00cf48c0d1f94208980c420496563a32"
format_stamp: "Formatted at 2026-08-06 18:38:11 on dist-test-slave-1xrh"
server_key: "f71c618ac91356286887637010bd6577"
server_key_iv: "e7aca7567388481ee9d3eebb5c0f55be"
server_key_version: "encryptionkey@0"
I20260806 18:38:11.254920 6202 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786041246094603-20232-0/minicluster-data/ts-0/wal/instance:
uuid: "00cf48c0d1f94208980c420496563a32"
format_stamp: "Formatted at 2026-08-06 18:38:11 on dist-test-slave-1xrh"
server_key: "f71c618ac91356286887637010bd6577"
server_key_iv: "e7aca7567388481ee9d3eebb5c0f55be"
server_key_version: "encryptionkey@0"
I20260806 18:38:11.258311 6202 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.002s sys 0.000s
I20260806 18:38:11.260581 6216 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 18:38:11.261662 6202 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260806 18:38:11.261787 6202 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786041246094603-20232-0/minicluster-data/ts-0/data,/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786041246094603-20232-0/minicluster-data/ts-0/wal
uuid: "00cf48c0d1f94208980c420496563a32"
format_stamp: "Formatted at 2026-08-06 18:38:11 on dist-test-slave-1xrh"
server_key: "f71c618ac91356286887637010bd6577"
server_key_iv: "e7aca7567388481ee9d3eebb5c0f55be"
server_key_version: "encryptionkey@0"
I20260806 18:38:11.261888 6202 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786041246094603-20232-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786041246094603-20232-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786041246094603-20232-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 18:38:11.280949 6202 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 18:38:11.281558 6202 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 18:38:11.281744 6202 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 18:38:11.282286 6202 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 18:38:11.283197 6202 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 18:38:11.283260 6202 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 18:38:11.283324 6202 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 18:38:11.283376 6202 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 18:38:11.292217 6202 rpc_server.cc:307] RPC server started. Bound to: 127.19.194.1:46001
I20260806 18:38:11.292260 6340 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.194.1:46001 every 8 connection(s)
I20260806 18:38:11.293015 6202 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786041246094603-20232-0/minicluster-data/ts-0/data/info.pb
I20260806 18:38:11.301954 6181 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:38:11.294726 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.194.1:58259 (local address 127.19.194.62:44161)
0806 18:38:11.294858 (+ 132us) server_negotiation.cc:206] Beginning negotiation
0806 18:38:11.294863 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 18:38:11.295809 (+ 946us) server_negotiation.cc:407] Connection header received
0806 18:38:11.297242 (+ 1433us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:38:11.297245 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:38:11.297312 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:38:11.297417 (+ 105us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:38:11.298261 (+ 844us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:38:11.298814 (+ 553us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:38:11.300072 (+ 1258us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:38:11.300303 (+ 231us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:38:11.300428 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:38:11.301446 (+ 1018us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:38:11.301450 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:38:11.301457 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 18:38:11.301468 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:38:11.301485 (+ 17us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 18:38:11.301493 (+ 8us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:38:11.301567 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0806 18:38:11.301728 (+ 161us) server_negotiation.cc:299] Negotiation successful
0806 18:38:11.301810 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260806 18:38:11.302539 20232 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJW69tn/build/debug/bin/kudu as pid 6202
I20260806 18:38:11.302685 20232 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786041246094603-20232-0/minicluster-data/ts-0/wal/instance
I20260806 18:38:11.302765 6343 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:38:11.295062 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.194.62:44161 (local address 127.19.194.1:58259)
0806 18:38:11.295670 (+ 608us) negotiation.cc:107] Waiting for socket to connect
0806 18:38:11.295707 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0806 18:38:11.297026 (+ 1319us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:38:11.297591 (+ 565us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:38:11.297598 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:38:11.297676 (+ 78us) client_negotiation.cc:189] Negotiated authn=SASL
0806 18:38:11.298121 (+ 445us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:38:11.298133 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:38:11.298974 (+ 841us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:38:11.298980 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:38:11.299947 (+ 967us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:38:11.299957 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:38:11.300471 (+ 514us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:38:11.300475 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:38:11.300631 (+ 156us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:38:11.301178 (+ 547us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0806 18:38:11.301180 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0806 18:38:11.301216 (+ 36us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0806 18:38:11.301274 (+ 58us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 18:38:11.301569 (+ 295us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 18:38:11.301582 (+ 13us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 18:38:11.301588 (+ 6us) client_negotiation.cc:769] Sending connection context
0806 18:38:11.301714 (+ 126us) client_negotiation.cc:240] Negotiation successful
0806 18:38:11.302006 (+ 292us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":256,"thread_start_us":107,"threads_started":1}
I20260806 18:38:11.303069 20232 external_mini_cluster.cc:1428] Setting key dd364ba0e3397c0242ad495a3a974f5d
I20260806 18:38:11.304073 6341 heartbeater.cc:347] Connected to a master server at 127.19.194.62:44161
I20260806 18:38:11.304306 6341 heartbeater.cc:464] Registering TS with master...
I20260806 18:38:11.304795 6341 heartbeater.cc:511] Master 127.19.194.62:44161 requested a full tablet report, sending...
I20260806 18:38:11.304988 20232 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJW69tn/build/debug/bin/kudu
/tmp/dist-test-taskJW69tn/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786041246094603-20232-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786041246094603-20232-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786041246094603-20232-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786041246094603-20232-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.19.194.2:0
--local_ip_for_outbound_sockets=127.19.194.2
--webserver_interface=127.19.194.2
--webserver_port=0
--tserver_master_addrs=127.19.194.62:44161
--builtin_ntp_servers=127.19.194.20:39149
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260806 18:38:11.306214 6139 ts_manager.cc:194] Registered new tserver with Master: 00cf48c0d1f94208980c420496563a32 (127.19.194.1:46001)
I20260806 18:38:11.307803 6139 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.19.194.1:58259
W20260806 18:38:11.420395 6344 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 18:38:11.420637 6344 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 18:38:11.420673 6344 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 18:38:11.424897 6344 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 18:38:11.424943 6344 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 18:38:11.425045 6344 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.19.194.2
I20260806 18:38:11.429692 6344 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.194.20:39149
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786041246094603-20232-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786041246094603-20232-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.19.194.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786041246094603-20232-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.19.194.2
--webserver_port=0
--tserver_master_addrs=127.19.194.62:44161
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6344
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.19.194.2
--log_dir=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786041246094603-20232-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 17:59:13 UTC on ccycloud.quasar-wvsnzx.root.comops.site
build id 4571028
I20260806 18:38:11.430657 6344 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 18:38:11.432015 6344 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 18:38:11.441707 6350 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:38:11.441718 6352 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:38:11.441738 6349 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 18:38:11.442046 6344 server_base.cc:1034] running on GCE node
I20260806 18:38:11.442443 6344 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 18:38:11.443226 6344 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 18:38:11.444401 6344 hybrid_clock.cc:648] HybridClock initialized: now 1786041491444373 us; error 43 us; skew 500 ppm
I20260806 18:38:11.446025 6344 webserver.cc:460] Webserver started at http://127.19.194.2:37543/ using document root <none> and password file <none>
I20260806 18:38:11.446514 6344 fs_manager.cc:359] Metadata directory not provided
I20260806 18:38:11.446569 6344 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 18:38:11.446749 6344 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 18:38:11.448734 6344 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786041246094603-20232-0/minicluster-data/ts-1/data/instance:
uuid: "88785628e18e4089a4bac523d6f56b78"
format_stamp: "Formatted at 2026-08-06 18:38:11 on dist-test-slave-1xrh"
server_key: "02adb1a5709184803711ff83abef7fd6"
server_key_iv: "52b18a3563237b949f0548e1f124a4a0"
server_key_version: "encryptionkey@0"
I20260806 18:38:11.449237 6344 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786041246094603-20232-0/minicluster-data/ts-1/wal/instance:
uuid: "88785628e18e4089a4bac523d6f56b78"
format_stamp: "Formatted at 2026-08-06 18:38:11 on dist-test-slave-1xrh"
server_key: "02adb1a5709184803711ff83abef7fd6"
server_key_iv: "52b18a3563237b949f0548e1f124a4a0"
server_key_version: "encryptionkey@0"
I20260806 18:38:11.452610 6344 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.002s sys 0.003s
I20260806 18:38:11.454723 6358 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 18:38:11.455888 6344 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260806 18:38:11.456018 6344 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786041246094603-20232-0/minicluster-data/ts-1/data,/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786041246094603-20232-0/minicluster-data/ts-1/wal
uuid: "88785628e18e4089a4bac523d6f56b78"
format_stamp: "Formatted at 2026-08-06 18:38:11 on dist-test-slave-1xrh"
server_key: "02adb1a5709184803711ff83abef7fd6"
server_key_iv: "52b18a3563237b949f0548e1f124a4a0"
server_key_version: "encryptionkey@0"
I20260806 18:38:11.456125 6344 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786041246094603-20232-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786041246094603-20232-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786041246094603-20232-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 18:38:11.467947 6344 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 18:38:11.468580 6344 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 18:38:11.468803 6344 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 18:38:11.469377 6344 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 18:38:11.470283 6344 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 18:38:11.470329 6344 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 18:38:11.470394 6344 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 18:38:11.470440 6344 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 18:38:11.478720 6344 rpc_server.cc:307] RPC server started. Bound to: 127.19.194.2:42081
I20260806 18:38:11.478796 6480 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.194.2:42081 every 8 connection(s)
I20260806 18:38:11.479600 6344 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786041246094603-20232-0/minicluster-data/ts-1/data/info.pb
I20260806 18:38:11.481765 20232 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJW69tn/build/debug/bin/kudu as pid 6344
I20260806 18:38:11.481884 20232 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786041246094603-20232-0/minicluster-data/ts-1/wal/instance
I20260806 18:38:11.482261 20232 external_mini_cluster.cc:1428] Setting key 28879b8f5abbaeaa1d3bd5a981c555fc
I20260806 18:38:11.484529 20232 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJW69tn/build/debug/bin/kudu
/tmp/dist-test-taskJW69tn/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786041246094603-20232-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786041246094603-20232-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786041246094603-20232-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786041246094603-20232-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.19.194.3:0
--local_ip_for_outbound_sockets=127.19.194.3
--webserver_interface=127.19.194.3
--webserver_port=0
--tserver_master_addrs=127.19.194.62:44161
--builtin_ntp_servers=127.19.194.20:39149
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260806 18:38:11.489293 6181 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:38:11.481345 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.194.2:35209 (local address 127.19.194.62:44161)
0806 18:38:11.481544 (+ 199us) server_negotiation.cc:206] Beginning negotiation
0806 18:38:11.481550 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 18:38:11.482727 (+ 1177us) server_negotiation.cc:407] Connection header received
0806 18:38:11.484442 (+ 1715us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:38:11.484445 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:38:11.484490 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:38:11.484559 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:38:11.485696 (+ 1137us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:38:11.486195 (+ 499us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:38:11.487388 (+ 1193us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:38:11.487626 (+ 238us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:38:11.487737 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:38:11.488809 (+ 1072us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:38:11.488813 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:38:11.488815 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 18:38:11.488835 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:38:11.488853 (+ 18us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 18:38:11.488860 (+ 7us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:38:11.488941 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0806 18:38:11.489094 (+ 153us) server_negotiation.cc:299] Negotiation successful
0806 18:38:11.489155 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":73}
I20260806 18:38:11.490053 6483 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:38:11.481728 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.194.62:44161 (local address 127.19.194.2:35209)
0806 18:38:11.482555 (+ 827us) negotiation.cc:107] Waiting for socket to connect
0806 18:38:11.482606 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0806 18:38:11.484211 (+ 1605us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:38:11.484762 (+ 551us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:38:11.484774 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:38:11.484886 (+ 112us) client_negotiation.cc:189] Negotiated authn=SASL
0806 18:38:11.485539 (+ 653us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:38:11.485561 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:38:11.486340 (+ 779us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:38:11.486343 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:38:11.487264 (+ 921us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:38:11.487274 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:38:11.487774 (+ 500us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:38:11.487778 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:38:11.487944 (+ 166us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:38:11.488554 (+ 610us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0806 18:38:11.488556 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0806 18:38:11.488594 (+ 38us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0806 18:38:11.488655 (+ 61us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 18:38:11.488943 (+ 288us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 18:38:11.488956 (+ 13us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 18:38:11.488963 (+ 7us) client_negotiation.cc:769] Sending connection context
0806 18:38:11.489092 (+ 129us) client_negotiation.cc:240] Negotiation successful
0806 18:38:11.489351 (+ 259us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":338,"thread_start_us":162,"threads_started":1}
I20260806 18:38:11.491271 6481 heartbeater.cc:347] Connected to a master server at 127.19.194.62:44161
I20260806 18:38:11.491490 6481 heartbeater.cc:464] Registering TS with master...
I20260806 18:38:11.491969 6481 heartbeater.cc:511] Master 127.19.194.62:44161 requested a full tablet report, sending...
I20260806 18:38:11.492977 6139 ts_manager.cc:194] Registered new tserver with Master: 88785628e18e4089a4bac523d6f56b78 (127.19.194.2:42081)
I20260806 18:38:11.493520 6139 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.19.194.2:35209
W20260806 18:38:11.593777 6484 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 18:38:11.594085 6484 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 18:38:11.594158 6484 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 18:38:11.598256 6484 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 18:38:11.598330 6484 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 18:38:11.598414 6484 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.19.194.3
I20260806 18:38:11.603097 6484 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.194.20:39149
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786041246094603-20232-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786041246094603-20232-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.19.194.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786041246094603-20232-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.19.194.3
--webserver_port=0
--tserver_master_addrs=127.19.194.62:44161
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6484
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.19.194.3
--log_dir=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786041246094603-20232-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 17:59:13 UTC on ccycloud.quasar-wvsnzx.root.comops.site
build id 4571028
I20260806 18:38:11.604137 6484 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 18:38:11.605436 6484 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 18:38:11.614586 6491 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 18:38:11.614652 6484 server_base.cc:1034] running on GCE node
W20260806 18:38:11.614652 6490 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:38:11.614697 6493 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 18:38:11.615265 6484 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 18:38:11.615854 6484 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 18:38:11.617022 6484 hybrid_clock.cc:648] HybridClock initialized: now 1786041491617004 us; error 29 us; skew 500 ppm
I20260806 18:38:11.618726 6484 webserver.cc:460] Webserver started at http://127.19.194.3:35225/ using document root <none> and password file <none>
I20260806 18:38:11.619271 6484 fs_manager.cc:359] Metadata directory not provided
I20260806 18:38:11.619334 6484 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 18:38:11.619493 6484 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 18:38:11.621483 6484 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786041246094603-20232-0/minicluster-data/ts-2/data/instance:
uuid: "e32e49bb89c844f79773cdbc1c57e529"
format_stamp: "Formatted at 2026-08-06 18:38:11 on dist-test-slave-1xrh"
server_key: "2211ca173fd059c2f7bd6c5fa79e706d"
server_key_iv: "4bccd6a9253009be999cde5b929e4547"
server_key_version: "encryptionkey@0"
I20260806 18:38:11.621965 6484 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786041246094603-20232-0/minicluster-data/ts-2/wal/instance:
uuid: "e32e49bb89c844f79773cdbc1c57e529"
format_stamp: "Formatted at 2026-08-06 18:38:11 on dist-test-slave-1xrh"
server_key: "2211ca173fd059c2f7bd6c5fa79e706d"
server_key_iv: "4bccd6a9253009be999cde5b929e4547"
server_key_version: "encryptionkey@0"
I20260806 18:38:11.625396 6484 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.002s sys 0.003s
I20260806 18:38:11.627614 6499 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 18:38:11.628697 6484 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260806 18:38:11.628784 6484 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786041246094603-20232-0/minicluster-data/ts-2/data,/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786041246094603-20232-0/minicluster-data/ts-2/wal
uuid: "e32e49bb89c844f79773cdbc1c57e529"
format_stamp: "Formatted at 2026-08-06 18:38:11 on dist-test-slave-1xrh"
server_key: "2211ca173fd059c2f7bd6c5fa79e706d"
server_key_iv: "4bccd6a9253009be999cde5b929e4547"
server_key_version: "encryptionkey@0"
I20260806 18:38:11.628861 6484 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786041246094603-20232-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786041246094603-20232-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786041246094603-20232-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 18:38:11.653054 6484 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 18:38:11.653647 6484 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 18:38:11.653784 6484 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 18:38:11.654268 6484 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 18:38:11.655110 6484 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 18:38:11.655215 6484 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 18:38:11.655256 6484 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 18:38:11.655277 6484 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 18:38:11.663954 6484 rpc_server.cc:307] RPC server started. Bound to: 127.19.194.3:45477
I20260806 18:38:11.663981 6622 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.194.3:45477 every 8 connection(s)
I20260806 18:38:11.664793 6484 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786041246094603-20232-0/minicluster-data/ts-2/data/info.pb
I20260806 18:38:11.671047 20232 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJW69tn/build/debug/bin/kudu as pid 6484
I20260806 18:38:11.671196 20232 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786041246094603-20232-0/minicluster-data/ts-2/wal/instance
I20260806 18:38:11.671502 20232 external_mini_cluster.cc:1428] Setting key 083be03d15fa73e8dd9746758db45a47
I20260806 18:38:11.674244 6181 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:38:11.666671 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.194.3:59677 (local address 127.19.194.62:44161)
0806 18:38:11.666811 (+ 140us) server_negotiation.cc:206] Beginning negotiation
0806 18:38:11.666815 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 18:38:11.667889 (+ 1074us) server_negotiation.cc:407] Connection header received
0806 18:38:11.669319 (+ 1430us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:38:11.669321 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:38:11.669369 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:38:11.669522 (+ 153us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:38:11.670289 (+ 767us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:38:11.670858 (+ 569us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:38:11.672232 (+ 1374us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:38:11.672512 (+ 280us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:38:11.672623 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:38:11.673755 (+ 1132us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:38:11.673759 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:38:11.673761 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 18:38:11.673773 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:38:11.673800 (+ 27us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 18:38:11.673808 (+ 8us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:38:11.673890 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0806 18:38:11.674052 (+ 162us) server_negotiation.cc:299] Negotiation successful
0806 18:38:11.674109 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260806 18:38:11.674975 6625 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:38:11.667012 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.194.62:44161 (local address 127.19.194.3:59677)
0806 18:38:11.667745 (+ 733us) negotiation.cc:107] Waiting for socket to connect
0806 18:38:11.667784 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0806 18:38:11.669139 (+ 1355us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:38:11.669646 (+ 507us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:38:11.669652 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:38:11.669731 (+ 79us) client_negotiation.cc:189] Negotiated authn=SASL
0806 18:38:11.670169 (+ 438us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:38:11.670180 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:38:11.670988 (+ 808us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:38:11.670991 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:38:11.672131 (+ 1140us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:38:11.672141 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:38:11.672691 (+ 550us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:38:11.672697 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:38:11.672876 (+ 179us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:38:11.673490 (+ 614us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0806 18:38:11.673492 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0806 18:38:11.673542 (+ 50us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0806 18:38:11.673612 (+ 70us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 18:38:11.673904 (+ 292us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 18:38:11.673915 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 18:38:11.673920 (+ 5us) client_negotiation.cc:769] Sending connection context
0806 18:38:11.674044 (+ 124us) client_negotiation.cc:240] Negotiation successful
0806 18:38:11.674289 (+ 245us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":295,"thread_start_us":158,"threads_started":1}
I20260806 18:38:11.675976 6623 heartbeater.cc:347] Connected to a master server at 127.19.194.62:44161
I20260806 18:38:11.676198 6623 heartbeater.cc:464] Registering TS with master...
I20260806 18:38:11.676658 6623 heartbeater.cc:511] Master 127.19.194.62:44161 requested a full tablet report, sending...
I20260806 18:38:11.677585 6139 ts_manager.cc:194] Registered new tserver with Master: e32e49bb89c844f79773cdbc1c57e529 (127.19.194.3:45477)
I20260806 18:38:11.678122 6139 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.19.194.3:59677
I20260806 18:38:11.685243 20232 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260806 18:38:11.690961 6181 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:38:11.687293 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35738 (local address 127.19.194.62:44161)
0806 18:38:11.687427 (+ 134us) server_negotiation.cc:206] Beginning negotiation
0806 18:38:11.687433 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 18:38:11.687636 (+ 203us) server_negotiation.cc:407] Connection header received
0806 18:38:11.687769 (+ 133us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:38:11.687772 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:38:11.687807 (+ 35us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:38:11.687885 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:38:11.688161 (+ 276us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:38:11.688643 (+ 482us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:38:11.689479 (+ 836us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:38:11.689704 (+ 225us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:38:11.689809 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:38:11.690534 (+ 725us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:38:11.690538 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:38:11.690542 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 18:38:11.690558 (+ 16us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:38:11.690574 (+ 16us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 18:38:11.690582 (+ 8us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:38:11.690659 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0806 18:38:11.690764 (+ 105us) server_negotiation.cc:299] Negotiation successful
0806 18:38:11.690810 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260806 18:38:11.693693 6139 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:35738:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260806 18:38:11.696794 6139 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 18:38:11.710449 6635 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:38:11.704874 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.194.1:46001 (local address 127.0.0.1:56230)
0806 18:38:11.705646 (+ 772us) negotiation.cc:107] Waiting for socket to connect
0806 18:38:11.705696 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0806 18:38:11.705864 (+ 168us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:38:11.706572 (+ 708us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:38:11.706578 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:38:11.706601 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 18:38:11.706727 (+ 126us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:38:11.706731 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:38:11.707921 (+ 1190us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:38:11.707933 (+ 12us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:38:11.709091 (+ 1158us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:38:11.709101 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:38:11.709958 (+ 857us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:38:11.709961 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:38:11.710062 (+ 101us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:38:11.710100 (+ 38us) client_negotiation.cc:769] Sending connection context
0806 18:38:11.710209 (+ 109us) client_negotiation.cc:240] Negotiation successful
0806 18:38:11.710314 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":603,"thread_start_us":67,"threads_started":1}
I20260806 18:38:11.710736 6636 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:38:11.705019 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56230 (local address 127.19.194.1:46001)
0806 18:38:11.706170 (+ 1151us) server_negotiation.cc:206] Beginning negotiation
0806 18:38:11.706181 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0806 18:38:11.706201 (+ 20us) server_negotiation.cc:407] Connection header received
0806 18:38:11.706265 (+ 64us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:38:11.706269 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:38:11.706414 (+ 145us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:38:11.706571 (+ 157us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 18:38:11.706836 (+ 265us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:38:11.707816 (+ 980us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:38:11.709227 (+ 1411us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:38:11.709848 (+ 621us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:38:11.709962 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:38:11.710132 (+ 170us) server_negotiation.cc:1035] Waiting for connection context
0806 18:38:11.710237 (+ 105us) server_negotiation.cc:299] Negotiation successful
0806 18:38:11.710328 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1044,"thread_start_us":152,"threads_started":1}
I20260806 18:38:11.711254 6634 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:38:11.704948 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.194.3:45477 (local address 127.0.0.1:41518)
0806 18:38:11.705945 (+ 997us) negotiation.cc:107] Waiting for socket to connect
0806 18:38:11.705958 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0806 18:38:11.706038 (+ 80us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:38:11.706746 (+ 708us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:38:11.706748 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:38:11.706765 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 18:38:11.706879 (+ 114us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:38:11.706888 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:38:11.708354 (+ 1466us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:38:11.708357 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:38:11.709579 (+ 1222us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:38:11.709589 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:38:11.710801 (+ 1212us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:38:11.710805 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:38:11.710900 (+ 95us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:38:11.710937 (+ 37us) client_negotiation.cc:769] Sending connection context
0806 18:38:11.711031 (+ 94us) client_negotiation.cc:240] Negotiation successful
0806 18:38:11.711097 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":916,"thread_start_us":101,"threads_started":1}
I20260806 18:38:11.711617 6638 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:38:11.705566 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.194.2:42081 (local address 127.0.0.1:44018)
0806 18:38:11.706433 (+ 867us) negotiation.cc:107] Waiting for socket to connect
0806 18:38:11.706446 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0806 18:38:11.706566 (+ 120us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:38:11.707239 (+ 673us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:38:11.707242 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:38:11.707262 (+ 20us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 18:38:11.707346 (+ 84us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:38:11.707351 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:38:11.708660 (+ 1309us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:38:11.708664 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:38:11.709766 (+ 1102us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:38:11.709773 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:38:11.711206 (+ 1433us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:38:11.711210 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:38:11.711283 (+ 73us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:38:11.711324 (+ 41us) client_negotiation.cc:769] Sending connection context
0806 18:38:11.711405 (+ 81us) client_negotiation.cc:240] Negotiation successful
0806 18:38:11.711479 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":793,"thread_start_us":131,"threads_started":1}
I20260806 18:38:11.711726 6637 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:38:11.705335 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41518 (local address 127.19.194.3:45477)
0806 18:38:11.705775 (+ 440us) server_negotiation.cc:206] Beginning negotiation
0806 18:38:11.705793 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0806 18:38:11.706132 (+ 339us) server_negotiation.cc:407] Connection header received
0806 18:38:11.706213 (+ 81us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:38:11.706220 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:38:11.706414 (+ 194us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:38:11.706571 (+ 157us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 18:38:11.707021 (+ 450us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:38:11.708206 (+ 1185us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:38:11.709716 (+ 1510us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:38:11.710457 (+ 741us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:38:11.710569 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:38:11.710734 (+ 165us) server_negotiation.cc:1035] Waiting for connection context
0806 18:38:11.711471 (+ 737us) server_negotiation.cc:299] Negotiation successful
0806 18:38:11.711573 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":323,"thread_start_us":139,"threads_started":1}
I20260806 18:38:11.712153 6639 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:38:11.706030 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44018 (local address 127.19.194.2:42081)
0806 18:38:11.706357 (+ 327us) server_negotiation.cc:206] Beginning negotiation
0806 18:38:11.706369 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0806 18:38:11.706828 (+ 459us) server_negotiation.cc:407] Connection header received
0806 18:38:11.706892 (+ 64us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:38:11.706895 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:38:11.707091 (+ 196us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:38:11.707261 (+ 170us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 18:38:11.707440 (+ 179us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:38:11.708540 (+ 1100us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:38:11.709877 (+ 1337us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:38:11.710819 (+ 942us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:38:11.710927 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:38:11.711108 (+ 181us) server_negotiation.cc:1035] Waiting for connection context
0806 18:38:11.711870 (+ 762us) server_negotiation.cc:299] Negotiation successful
0806 18:38:11.711996 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":224,"thread_start_us":135,"threads_started":1}
I20260806 18:38:11.713739 6269 tablet_service.cc:1467] Processing CreateTablet for tablet d0f38169e26d468cb71c05c1538af201 (DEFAULT_TABLE table=test-table [id=8e749598ad814fd5b47d2c399c9396a1]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 18:38:11.714154 6408 tablet_service.cc:1467] Processing CreateTablet for tablet d0f38169e26d468cb71c05c1538af201 (DEFAULT_TABLE table=test-table [id=8e749598ad814fd5b47d2c399c9396a1]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 18:38:11.714308 6551 tablet_service.cc:1467] Processing CreateTablet for tablet d0f38169e26d468cb71c05c1538af201 (DEFAULT_TABLE table=test-table [id=8e749598ad814fd5b47d2c399c9396a1]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 18:38:11.714847 6269 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d0f38169e26d468cb71c05c1538af201. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 18:38:11.715104 6408 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d0f38169e26d468cb71c05c1538af201. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 18:38:11.715282 6551 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d0f38169e26d468cb71c05c1538af201. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 18:38:11.721181 6640 tablet_bootstrap.cc:492] T d0f38169e26d468cb71c05c1538af201 P 88785628e18e4089a4bac523d6f56b78: Bootstrap starting.
I20260806 18:38:11.722580 6642 tablet_bootstrap.cc:492] T d0f38169e26d468cb71c05c1538af201 P e32e49bb89c844f79773cdbc1c57e529: Bootstrap starting.
I20260806 18:38:11.723325 6641 tablet_bootstrap.cc:492] T d0f38169e26d468cb71c05c1538af201 P 00cf48c0d1f94208980c420496563a32: Bootstrap starting.
I20260806 18:38:11.724010 6640 tablet_bootstrap.cc:654] T d0f38169e26d468cb71c05c1538af201 P 88785628e18e4089a4bac523d6f56b78: Neither blocks nor log segments found. Creating new log.
I20260806 18:38:11.724781 6642 tablet_bootstrap.cc:654] T d0f38169e26d468cb71c05c1538af201 P e32e49bb89c844f79773cdbc1c57e529: Neither blocks nor log segments found. Creating new log.
I20260806 18:38:11.724915 6640 log.cc:824] T d0f38169e26d468cb71c05c1538af201 P 88785628e18e4089a4bac523d6f56b78: Log is configured to *not* fsync() on all Append() calls
I20260806 18:38:11.725133 6641 tablet_bootstrap.cc:654] T d0f38169e26d468cb71c05c1538af201 P 00cf48c0d1f94208980c420496563a32: Neither blocks nor log segments found. Creating new log.
I20260806 18:38:11.725553 6642 log.cc:824] T d0f38169e26d468cb71c05c1538af201 P e32e49bb89c844f79773cdbc1c57e529: Log is configured to *not* fsync() on all Append() calls
I20260806 18:38:11.725940 6641 log.cc:824] T d0f38169e26d468cb71c05c1538af201 P 00cf48c0d1f94208980c420496563a32: Log is configured to *not* fsync() on all Append() calls
I20260806 18:38:11.727428 6640 tablet_bootstrap.cc:492] T d0f38169e26d468cb71c05c1538af201 P 88785628e18e4089a4bac523d6f56b78: No bootstrap required, opened a new log
I20260806 18:38:11.727440 6642 tablet_bootstrap.cc:492] T d0f38169e26d468cb71c05c1538af201 P e32e49bb89c844f79773cdbc1c57e529: No bootstrap required, opened a new log
I20260806 18:38:11.727627 6642 ts_tablet_manager.cc:1397] T d0f38169e26d468cb71c05c1538af201 P e32e49bb89c844f79773cdbc1c57e529: Time spent bootstrapping tablet: real 0.005s user 0.004s sys 0.000s
I20260806 18:38:11.727630 6640 ts_tablet_manager.cc:1397] T d0f38169e26d468cb71c05c1538af201 P 88785628e18e4089a4bac523d6f56b78: Time spent bootstrapping tablet: real 0.007s user 0.005s sys 0.000s
I20260806 18:38:11.729236 6641 tablet_bootstrap.cc:492] T d0f38169e26d468cb71c05c1538af201 P 00cf48c0d1f94208980c420496563a32: No bootstrap required, opened a new log
I20260806 18:38:11.729409 6641 ts_tablet_manager.cc:1397] T d0f38169e26d468cb71c05c1538af201 P 00cf48c0d1f94208980c420496563a32: Time spent bootstrapping tablet: real 0.007s user 0.003s sys 0.000s
I20260806 18:38:11.730683 6642 raft_consensus.cc:348] T d0f38169e26d468cb71c05c1538af201 P e32e49bb89c844f79773cdbc1c57e529 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "00cf48c0d1f94208980c420496563a32" member_type: VOTER last_known_addr { host: "127.19.194.1" port: 46001 } } peers { permanent_uuid: "e32e49bb89c844f79773cdbc1c57e529" member_type: VOTER last_known_addr { host: "127.19.194.3" port: 45477 } } peers { permanent_uuid: "88785628e18e4089a4bac523d6f56b78" member_type: VOTER last_known_addr { host: "127.19.194.2" port: 42081 } }
I20260806 18:38:11.730906 6642 raft_consensus.cc:374] T d0f38169e26d468cb71c05c1538af201 P e32e49bb89c844f79773cdbc1c57e529 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 18:38:11.730974 6642 raft_consensus.cc:729] T d0f38169e26d468cb71c05c1538af201 P e32e49bb89c844f79773cdbc1c57e529 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e32e49bb89c844f79773cdbc1c57e529, State: Initialized, Role: FOLLOWER
I20260806 18:38:11.731498 6642 consensus_queue.cc:260] T d0f38169e26d468cb71c05c1538af201 P e32e49bb89c844f79773cdbc1c57e529 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "00cf48c0d1f94208980c420496563a32" member_type: VOTER last_known_addr { host: "127.19.194.1" port: 46001 } } peers { permanent_uuid: "e32e49bb89c844f79773cdbc1c57e529" member_type: VOTER last_known_addr { host: "127.19.194.3" port: 45477 } } peers { permanent_uuid: "88785628e18e4089a4bac523d6f56b78" member_type: VOTER last_known_addr { host: "127.19.194.2" port: 42081 } }
I20260806 18:38:11.731905 6640 raft_consensus.cc:348] T d0f38169e26d468cb71c05c1538af201 P 88785628e18e4089a4bac523d6f56b78 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "00cf48c0d1f94208980c420496563a32" member_type: VOTER last_known_addr { host: "127.19.194.1" port: 46001 } } peers { permanent_uuid: "e32e49bb89c844f79773cdbc1c57e529" member_type: VOTER last_known_addr { host: "127.19.194.3" port: 45477 } } peers { permanent_uuid: "88785628e18e4089a4bac523d6f56b78" member_type: VOTER last_known_addr { host: "127.19.194.2" port: 42081 } }
I20260806 18:38:11.732139 6640 raft_consensus.cc:374] T d0f38169e26d468cb71c05c1538af201 P 88785628e18e4089a4bac523d6f56b78 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 18:38:11.732136 6623 heartbeater.cc:503] Master 127.19.194.62:44161 was elected leader, sending a full tablet report...
I20260806 18:38:11.732195 6640 raft_consensus.cc:729] T d0f38169e26d468cb71c05c1538af201 P 88785628e18e4089a4bac523d6f56b78 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 88785628e18e4089a4bac523d6f56b78, State: Initialized, Role: FOLLOWER
I20260806 18:38:11.732518 6642 ts_tablet_manager.cc:1428] T d0f38169e26d468cb71c05c1538af201 P e32e49bb89c844f79773cdbc1c57e529: Time spent starting tablet: real 0.005s user 0.005s sys 0.000s
I20260806 18:38:11.732645 6640 consensus_queue.cc:260] T d0f38169e26d468cb71c05c1538af201 P 88785628e18e4089a4bac523d6f56b78 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "00cf48c0d1f94208980c420496563a32" member_type: VOTER last_known_addr { host: "127.19.194.1" port: 46001 } } peers { permanent_uuid: "e32e49bb89c844f79773cdbc1c57e529" member_type: VOTER last_known_addr { host: "127.19.194.3" port: 45477 } } peers { permanent_uuid: "88785628e18e4089a4bac523d6f56b78" member_type: VOTER last_known_addr { host: "127.19.194.2" port: 42081 } }
I20260806 18:38:11.733163 6641 raft_consensus.cc:348] T d0f38169e26d468cb71c05c1538af201 P 00cf48c0d1f94208980c420496563a32 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "00cf48c0d1f94208980c420496563a32" member_type: VOTER last_known_addr { host: "127.19.194.1" port: 46001 } } peers { permanent_uuid: "e32e49bb89c844f79773cdbc1c57e529" member_type: VOTER last_known_addr { host: "127.19.194.3" port: 45477 } } peers { permanent_uuid: "88785628e18e4089a4bac523d6f56b78" member_type: VOTER last_known_addr { host: "127.19.194.2" port: 42081 } }
I20260806 18:38:11.733405 6641 raft_consensus.cc:374] T d0f38169e26d468cb71c05c1538af201 P 00cf48c0d1f94208980c420496563a32 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 18:38:11.733469 6641 raft_consensus.cc:729] T d0f38169e26d468cb71c05c1538af201 P 00cf48c0d1f94208980c420496563a32 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 00cf48c0d1f94208980c420496563a32, State: Initialized, Role: FOLLOWER
I20260806 18:38:11.733482 6481 heartbeater.cc:503] Master 127.19.194.62:44161 was elected leader, sending a full tablet report...
I20260806 18:38:11.733858 6641 consensus_queue.cc:260] T d0f38169e26d468cb71c05c1538af201 P 00cf48c0d1f94208980c420496563a32 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "00cf48c0d1f94208980c420496563a32" member_type: VOTER last_known_addr { host: "127.19.194.1" port: 46001 } } peers { permanent_uuid: "e32e49bb89c844f79773cdbc1c57e529" member_type: VOTER last_known_addr { host: "127.19.194.3" port: 45477 } } peers { permanent_uuid: "88785628e18e4089a4bac523d6f56b78" member_type: VOTER last_known_addr { host: "127.19.194.2" port: 42081 } }
I20260806 18:38:11.734807 6641 ts_tablet_manager.cc:1428] T d0f38169e26d468cb71c05c1538af201 P 00cf48c0d1f94208980c420496563a32: Time spent starting tablet: real 0.005s user 0.006s sys 0.000s
I20260806 18:38:11.735462 6341 heartbeater.cc:503] Master 127.19.194.62:44161 was elected leader, sending a full tablet report...
I20260806 18:38:11.736508 6640 ts_tablet_manager.cc:1428] T d0f38169e26d468cb71c05c1538af201 P 88785628e18e4089a4bac523d6f56b78: Time spent starting tablet: real 0.009s user 0.006s sys 0.000s
W20260806 18:38:11.794227 6342 tablet.cc:2369] T d0f38169e26d468cb71c05c1538af201 P 00cf48c0d1f94208980c420496563a32: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 18:38:11.886699 6648 raft_consensus.cc:482] T d0f38169e26d468cb71c05c1538af201 P 00cf48c0d1f94208980c420496563a32 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 18:38:11.887004 6648 raft_consensus.cc:504] T d0f38169e26d468cb71c05c1538af201 P 00cf48c0d1f94208980c420496563a32 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "00cf48c0d1f94208980c420496563a32" member_type: VOTER last_known_addr { host: "127.19.194.1" port: 46001 } } peers { permanent_uuid: "e32e49bb89c844f79773cdbc1c57e529" member_type: VOTER last_known_addr { host: "127.19.194.3" port: 45477 } } peers { permanent_uuid: "88785628e18e4089a4bac523d6f56b78" member_type: VOTER last_known_addr { host: "127.19.194.2" port: 42081 } }
I20260806 18:38:11.888237 6648 leader_election.cc:288] T d0f38169e26d468cb71c05c1538af201 P 00cf48c0d1f94208980c420496563a32 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e32e49bb89c844f79773cdbc1c57e529 (127.19.194.3:45477), 88785628e18e4089a4bac523d6f56b78 (127.19.194.2:42081)
I20260806 18:38:11.893600 6637 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:38:11.888559 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.194.1:46793 (local address 127.19.194.3:45477)
0806 18:38:11.888811 (+ 252us) server_negotiation.cc:206] Beginning negotiation
0806 18:38:11.888816 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 18:38:11.888898 (+ 82us) server_negotiation.cc:407] Connection header received
0806 18:38:11.889068 (+ 170us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:38:11.889071 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:38:11.889139 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:38:11.889240 (+ 101us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 18:38:11.889646 (+ 406us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:38:11.890424 (+ 778us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:38:11.892344 (+ 1920us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:38:11.892957 (+ 613us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:38:11.893064 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:38:11.893112 (+ 48us) server_negotiation.cc:1035] Waiting for connection context
0806 18:38:11.893404 (+ 292us) server_negotiation.cc:299] Negotiation successful
0806 18:38:11.893474 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":128}
I20260806 18:38:11.893600 6639 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:38:11.888630 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.194.1:37651 (local address 127.19.194.2:42081)
0806 18:38:11.888827 (+ 197us) server_negotiation.cc:206] Beginning negotiation
0806 18:38:11.888834 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 18:38:11.888975 (+ 141us) server_negotiation.cc:407] Connection header received
0806 18:38:11.889068 (+ 93us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:38:11.889071 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:38:11.889138 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:38:11.889240 (+ 102us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 18:38:11.889646 (+ 406us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:38:11.890433 (+ 787us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:38:11.892344 (+ 1911us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:38:11.892937 (+ 593us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:38:11.893064 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:38:11.893112 (+ 48us) server_negotiation.cc:1035] Waiting for connection context
0806 18:38:11.893404 (+ 292us) server_negotiation.cc:299] Negotiation successful
0806 18:38:11.893479 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":110}
I20260806 18:38:11.893666 6649 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:38:11.888526 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.194.2:42081 (local address 127.19.194.1:37651)
0806 18:38:11.888885 (+ 359us) negotiation.cc:107] Waiting for socket to connect
0806 18:38:11.888897 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0806 18:38:11.888970 (+ 73us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:38:11.889276 (+ 306us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:38:11.889279 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:38:11.889321 (+ 42us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 18:38:11.889526 (+ 205us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:38:11.889535 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:38:11.890553 (+ 1018us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:38:11.890557 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:38:11.892202 (+ 1645us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:38:11.892209 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:38:11.893073 (+ 864us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:38:11.893076 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:38:11.893216 (+ 140us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:38:11.893294 (+ 78us) client_negotiation.cc:769] Sending connection context
0806 18:38:11.893373 (+ 79us) client_negotiation.cc:240] Negotiation successful
0806 18:38:11.893481 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":271,"mutex_wait_us":237,"spinlock_wait_cycles":61696,"thread_start_us":63,"threads_started":1}
I20260806 18:38:11.893666 6650 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:38:11.888447 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.194.3:45477 (local address 127.19.194.1:46793)
0806 18:38:11.888787 (+ 340us) negotiation.cc:107] Waiting for socket to connect
0806 18:38:11.888803 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0806 18:38:11.888960 (+ 157us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:38:11.889276 (+ 316us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:38:11.889279 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:38:11.889320 (+ 41us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 18:38:11.889534 (+ 214us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:38:11.889539 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:38:11.890553 (+ 1014us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:38:11.890557 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:38:11.892196 (+ 1639us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:38:11.892209 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:38:11.893073 (+ 864us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:38:11.893076 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:38:11.893241 (+ 165us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:38:11.893301 (+ 60us) client_negotiation.cc:769] Sending connection context
0806 18:38:11.893373 (+ 72us) client_negotiation.cc:240] Negotiation successful
0806 18:38:11.893481 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":229,"mutex_wait_us":347,"spinlock_wait_cycles":28288,"thread_start_us":108,"threads_started":1}
I20260806 18:38:11.894307 6430 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d0f38169e26d468cb71c05c1538af201" candidate_uuid: "00cf48c0d1f94208980c420496563a32" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "88785628e18e4089a4bac523d6f56b78" is_pre_election: true
I20260806 18:38:11.894323 6573 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d0f38169e26d468cb71c05c1538af201" candidate_uuid: "00cf48c0d1f94208980c420496563a32" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e32e49bb89c844f79773cdbc1c57e529" is_pre_election: true
I20260806 18:38:11.894665 6573 raft_consensus.cc:2454] T d0f38169e26d468cb71c05c1538af201 P e32e49bb89c844f79773cdbc1c57e529 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 00cf48c0d1f94208980c420496563a32 in term 0.
I20260806 18:38:11.894672 6430 raft_consensus.cc:2454] T d0f38169e26d468cb71c05c1538af201 P 88785628e18e4089a4bac523d6f56b78 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 00cf48c0d1f94208980c420496563a32 in term 0.
I20260806 18:38:11.895174 6218 leader_election.cc:302] T d0f38169e26d468cb71c05c1538af201 P 00cf48c0d1f94208980c420496563a32 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 00cf48c0d1f94208980c420496563a32, e32e49bb89c844f79773cdbc1c57e529; no voters:
I20260806 18:38:11.895438 6648 raft_consensus.cc:2781] T d0f38169e26d468cb71c05c1538af201 P 00cf48c0d1f94208980c420496563a32 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 18:38:11.895524 6648 raft_consensus.cc:482] T d0f38169e26d468cb71c05c1538af201 P 00cf48c0d1f94208980c420496563a32 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 18:38:11.895555 6648 raft_consensus.cc:3037] T d0f38169e26d468cb71c05c1538af201 P 00cf48c0d1f94208980c420496563a32 [term 0 FOLLOWER]: Advancing to term 1
I20260806 18:38:11.896600 6648 raft_consensus.cc:504] T d0f38169e26d468cb71c05c1538af201 P 00cf48c0d1f94208980c420496563a32 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "00cf48c0d1f94208980c420496563a32" member_type: VOTER last_known_addr { host: "127.19.194.1" port: 46001 } } peers { permanent_uuid: "e32e49bb89c844f79773cdbc1c57e529" member_type: VOTER last_known_addr { host: "127.19.194.3" port: 45477 } } peers { permanent_uuid: "88785628e18e4089a4bac523d6f56b78" member_type: VOTER last_known_addr { host: "127.19.194.2" port: 42081 } }
I20260806 18:38:11.896978 6648 leader_election.cc:288] T d0f38169e26d468cb71c05c1538af201 P 00cf48c0d1f94208980c420496563a32 [CANDIDATE]: Term 1 election: Requested vote from peers e32e49bb89c844f79773cdbc1c57e529 (127.19.194.3:45477), 88785628e18e4089a4bac523d6f56b78 (127.19.194.2:42081)
I20260806 18:38:11.897327 6573 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d0f38169e26d468cb71c05c1538af201" candidate_uuid: "00cf48c0d1f94208980c420496563a32" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e32e49bb89c844f79773cdbc1c57e529"
I20260806 18:38:11.897332 6430 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d0f38169e26d468cb71c05c1538af201" candidate_uuid: "00cf48c0d1f94208980c420496563a32" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "88785628e18e4089a4bac523d6f56b78"
I20260806 18:38:11.897464 6573 raft_consensus.cc:3037] T d0f38169e26d468cb71c05c1538af201 P e32e49bb89c844f79773cdbc1c57e529 [term 0 FOLLOWER]: Advancing to term 1
I20260806 18:38:11.897496 6430 raft_consensus.cc:3037] T d0f38169e26d468cb71c05c1538af201 P 88785628e18e4089a4bac523d6f56b78 [term 0 FOLLOWER]: Advancing to term 1
I20260806 18:38:11.898486 6430 raft_consensus.cc:2454] T d0f38169e26d468cb71c05c1538af201 P 88785628e18e4089a4bac523d6f56b78 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 00cf48c0d1f94208980c420496563a32 in term 1.
I20260806 18:38:11.898525 6573 raft_consensus.cc:2454] T d0f38169e26d468cb71c05c1538af201 P e32e49bb89c844f79773cdbc1c57e529 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 00cf48c0d1f94208980c420496563a32 in term 1.
I20260806 18:38:11.898790 6220 leader_election.cc:302] T d0f38169e26d468cb71c05c1538af201 P 00cf48c0d1f94208980c420496563a32 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 00cf48c0d1f94208980c420496563a32, 88785628e18e4089a4bac523d6f56b78; no voters:
I20260806 18:38:11.898968 6648 raft_consensus.cc:2781] T d0f38169e26d468cb71c05c1538af201 P 00cf48c0d1f94208980c420496563a32 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 18:38:11.899230 6648 raft_consensus.cc:686] T d0f38169e26d468cb71c05c1538af201 P 00cf48c0d1f94208980c420496563a32 [term 1 LEADER]: Becoming Leader. State: Replica: 00cf48c0d1f94208980c420496563a32, State: Running, Role: LEADER
I20260806 18:38:11.899550 6648 consensus_queue.cc:237] T d0f38169e26d468cb71c05c1538af201 P 00cf48c0d1f94208980c420496563a32 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "00cf48c0d1f94208980c420496563a32" member_type: VOTER last_known_addr { host: "127.19.194.1" port: 46001 } } peers { permanent_uuid: "e32e49bb89c844f79773cdbc1c57e529" member_type: VOTER last_known_addr { host: "127.19.194.3" port: 45477 } } peers { permanent_uuid: "88785628e18e4089a4bac523d6f56b78" member_type: VOTER last_known_addr { host: "127.19.194.2" port: 42081 } }
I20260806 18:38:11.902501 6136 catalog_manager.cc:5495] T d0f38169e26d468cb71c05c1538af201 P 00cf48c0d1f94208980c420496563a32 reported cstate change: term changed from 0 to 1, leader changed from <none> to 00cf48c0d1f94208980c420496563a32 (127.19.194.1). New cstate: current_term: 1 leader_uuid: "00cf48c0d1f94208980c420496563a32" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "00cf48c0d1f94208980c420496563a32" member_type: VOTER last_known_addr { host: "127.19.194.1" port: 46001 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e32e49bb89c844f79773cdbc1c57e529" member_type: VOTER last_known_addr { host: "127.19.194.3" port: 45477 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "88785628e18e4089a4bac523d6f56b78" member_type: VOTER last_known_addr { host: "127.19.194.2" port: 42081 } health_report { overall_health: UNKNOWN } } }
W20260806 18:38:11.915731 6624 tablet.cc:2369] T d0f38169e26d468cb71c05c1538af201 P e32e49bb89c844f79773cdbc1c57e529: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 18:38:11.946103 6636 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:38:11.942560 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56236 (local address 127.19.194.1:46001)
0806 18:38:11.942707 (+ 147us) server_negotiation.cc:206] Beginning negotiation
0806 18:38:11.942712 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 18:38:11.942728 (+ 16us) server_negotiation.cc:407] Connection header received
0806 18:38:11.942822 (+ 94us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:38:11.942824 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:38:11.942893 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:38:11.943008 (+ 115us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 18:38:11.943296 (+ 288us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:38:11.943802 (+ 506us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:38:11.944675 (+ 873us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:38:11.944896 (+ 221us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:38:11.945013 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:38:11.945320 (+ 307us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 18:38:11.945701 (+ 381us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 18:38:11.945834 (+ 133us) server_negotiation.cc:1035] Waiting for connection context
0806 18:38:11.945898 (+ 64us) server_negotiation.cc:299] Negotiation successful
0806 18:38:11.945957 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260806 18:38:11.951694 6430 raft_consensus.cc:1261] T d0f38169e26d468cb71c05c1538af201 P 88785628e18e4089a4bac523d6f56b78 [term 1 FOLLOWER]: Refusing update from remote peer 00cf48c0d1f94208980c420496563a32: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 18:38:11.951694 6573 raft_consensus.cc:1261] T d0f38169e26d468cb71c05c1538af201 P e32e49bb89c844f79773cdbc1c57e529 [term 1 FOLLOWER]: Refusing update from remote peer 00cf48c0d1f94208980c420496563a32: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 18:38:11.952376 6648 consensus_queue.cc:1035] T d0f38169e26d468cb71c05c1538af201 P 00cf48c0d1f94208980c420496563a32 [LEADER]: Connected to new peer: Peer: permanent_uuid: "88785628e18e4089a4bac523d6f56b78" member_type: VOTER last_known_addr { host: "127.19.194.2" port: 42081 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 18:38:11.952613 6651 consensus_queue.cc:1035] T d0f38169e26d468cb71c05c1538af201 P 00cf48c0d1f94208980c420496563a32 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e32e49bb89c844f79773cdbc1c57e529" member_type: VOTER last_known_addr { host: "127.19.194.3" port: 45477 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 18:38:11.961211 6656 mvcc.cc:205] Tried to move back new op lower bound from 7315625951027138560 to 7315625950823030784. Current Snapshot: MvccSnapshot[applied={T|T < 7315625951027138560}]
I20260806 18:38:11.963413 6657 mvcc.cc:205] Tried to move back new op lower bound from 7315625951027138560 to 7315625950823030784. Current Snapshot: MvccSnapshot[applied={T|T < 7315625951027138560 or (T in {7315625951027138560})}]
I20260806 18:38:11.967491 6138 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:35738:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 18:38:11.967620 6138 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:35738:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 18:38:11.970088 6138 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 14d8bae9d1964184bca110daa4f57d1b: Sending DeleteTablet for 3 replicas of tablet d0f38169e26d468cb71c05c1538af201
I20260806 18:38:11.970788 6269 tablet_service.cc:1514] Processing DeleteTablet for tablet d0f38169e26d468cb71c05c1538af201 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 18:38:11 UTC) from {username='slave'} at 127.0.0.1:56230
I20260806 18:38:11.970969 6551 tablet_service.cc:1514] Processing DeleteTablet for tablet d0f38169e26d468cb71c05c1538af201 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 18:38:11 UTC) from {username='slave'} at 127.0.0.1:41518
I20260806 18:38:11.971310 6408 tablet_service.cc:1514] Processing DeleteTablet for tablet d0f38169e26d468cb71c05c1538af201 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 18:38:11 UTC) from {username='slave'} at 127.0.0.1:44018
I20260806 18:38:11.971570 20232 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJW69tn/build/debug/bin/kudu with pid 6202
I20260806 18:38:11.972450 6666 tablet_replica.cc:330] T d0f38169e26d468cb71c05c1538af201 P 88785628e18e4089a4bac523d6f56b78: stopping tablet replica
I20260806 18:38:11.972754 6666 raft_consensus.cc:2229] T d0f38169e26d468cb71c05c1538af201 P 88785628e18e4089a4bac523d6f56b78 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 18:38:11.973101 6666 raft_consensus.cc:2258] T d0f38169e26d468cb71c05c1538af201 P 88785628e18e4089a4bac523d6f56b78 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 18:38:11.973387 6665 tablet_replica.cc:330] T d0f38169e26d468cb71c05c1538af201 P e32e49bb89c844f79773cdbc1c57e529: stopping tablet replica
I20260806 18:38:11.973661 6665 raft_consensus.cc:2229] T d0f38169e26d468cb71c05c1538af201 P e32e49bb89c844f79773cdbc1c57e529 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 18:38:11.973913 6665 raft_consensus.cc:2258] T d0f38169e26d468cb71c05c1538af201 P e32e49bb89c844f79773cdbc1c57e529 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 18:38:11.975317 6665 ts_tablet_manager.cc:1905] T d0f38169e26d468cb71c05c1538af201 P e32e49bb89c844f79773cdbc1c57e529: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 18:38:11.975878 6666 ts_tablet_manager.cc:1905] T d0f38169e26d468cb71c05c1538af201 P 88785628e18e4089a4bac523d6f56b78: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 18:38:11.977720 6665 ts_tablet_manager.cc:1918] T d0f38169e26d468cb71c05c1538af201 P e32e49bb89c844f79773cdbc1c57e529: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 18:38:11.977829 6665 log.cc:1196] T d0f38169e26d468cb71c05c1538af201 P e32e49bb89c844f79773cdbc1c57e529: Deleting WAL directory at /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786041246094603-20232-0/minicluster-data/ts-2/wal/wals/d0f38169e26d468cb71c05c1538af201
I20260806 18:38:11.978125 6665 ts_tablet_manager.cc:1939] T d0f38169e26d468cb71c05c1538af201 P e32e49bb89c844f79773cdbc1c57e529: Deleting consensus metadata
I20260806 18:38:11.979204 6121 catalog_manager.cc:4841] TS e32e49bb89c844f79773cdbc1c57e529 (127.19.194.3:45477): tablet d0f38169e26d468cb71c05c1538af201 (table test-table [id=8e749598ad814fd5b47d2c399c9396a1]) successfully deleted
I20260806 18:38:11.979271 6666 ts_tablet_manager.cc:1918] T d0f38169e26d468cb71c05c1538af201 P 88785628e18e4089a4bac523d6f56b78: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 18:38:11.979360 6666 log.cc:1196] T d0f38169e26d468cb71c05c1538af201 P 88785628e18e4089a4bac523d6f56b78: Deleting WAL directory at /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786041246094603-20232-0/minicluster-data/ts-1/wal/wals/d0f38169e26d468cb71c05c1538af201
I20260806 18:38:11.979674 6666 ts_tablet_manager.cc:1939] T d0f38169e26d468cb71c05c1538af201 P 88785628e18e4089a4bac523d6f56b78: Deleting consensus metadata
I20260806 18:38:11.980789 6123 catalog_manager.cc:4841] TS 88785628e18e4089a4bac523d6f56b78 (127.19.194.2:42081): tablet d0f38169e26d468cb71c05c1538af201 (table test-table [id=8e749598ad814fd5b47d2c399c9396a1]) successfully deleted
W20260806 18:38:11.982132 6123 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.19.194.1:46001 (error 108)
I20260806 18:38:11.982343 20232 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJW69tn/build/debug/bin/kudu with pid 6344
I20260806 18:38:11.983623 6634 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:38:11.983251 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.194.1:46001 (local address 127.0.0.1:56250)
0806 18:38:11.983438 (+ 187us) negotiation.cc:107] Waiting for socket to connect
0806 18:38:11.983517 (+ 79us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.194.1:46001: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":95}
W20260806 18:38:11.983805 6123 catalog_manager.cc:4584] TS 00cf48c0d1f94208980c420496563a32 (127.19.194.1:46001): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet d0f38169e26d468cb71c05c1538af201: Network error: Client connection negotiation failed: client connection to 127.19.194.1:46001: connect: Connection refused (error 111)
I20260806 18:38:11.989528 20232 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJW69tn/build/debug/bin/kudu with pid 6484
I20260806 18:38:11.996553 20232 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJW69tn/build/debug/bin/kudu with pid 6104
2026-08-06T18:38:11Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/1 (1155 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/2
Loading random data
Initializing database '/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786041246094603-20232-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 18:38:12 dist-test-slave-1xrh krb5kdc[6670](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 18:38:12 dist-test-slave-1xrh krb5kdc[6670](info): set up 2 sockets
Aug 06 18:38:12 dist-test-slave-1xrh krb5kdc[6670](info): commencing operation
krb5kdc: starting...
W20260806 18:38:13.060748 20232 mini_kdc.cc:121] Time spent starting KDC: real 1.042s user 0.001s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786041246094603-20232-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786041246094603-20232-0/krb5kdc/test-user.keytab.
Aug 06 18:38:13 dist-test-slave-1xrh krb5kdc[6670](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041493, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T18:38:13Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T18:38:13Z Disabled control of system clock
WARNING: no policy specified for kudu/127.19.194.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.19.194.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.19.194.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786041246094603-20232-0/krb5kdc/kudu_127.19.194.62.keytab.
Entry for principal kudu/127.19.194.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786041246094603-20232-0/krb5kdc/kudu_127.19.194.62.keytab.
I20260806 18:38:13.181361 20232 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJW69tn/build/debug/bin/kudu
/tmp/dist-test-taskJW69tn/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786041246094603-20232-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786041246094603-20232-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786041246094603-20232-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786041246094603-20232-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.19.194.62:40041
--webserver_interface=127.19.194.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786041246094603-20232-0/krb5kdc/kudu_127.19.194.62.keytab
--principal=kudu/127.19.194.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.19.194.20:37085
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.19.194.62:40041
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786041246094603-20232-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786041246094603-20232-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786041246094603-20232-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 18:38:13.288533 6684 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 18:38:13.288765 6684 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 18:38:13.288801 6684 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 18:38:13.292620 6684 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 18:38:13.292667 6684 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 18:38:13.292685 6684 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 18:38:13.292701 6684 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 18:38:13.292716 6684 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 18:38:13.297590 6684 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.194.20:37085
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786041246094603-20232-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786041246094603-20232-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.19.194.62:40041
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786041246094603-20232-0/krb5kdc/kudu_127.19.194.62.keytab
--principal=kudu/127.19.194.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.19.194.62:40041
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786041246094603-20232-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.19.194.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6684
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786041246094603-20232-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 17:59:13 UTC on ccycloud.quasar-wvsnzx.root.comops.site
build id 4571028
I20260806 18:38:13.298506 6684 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 18:38:13.299515 6684 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 18:38:13.307602 6691 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:38:13.307611 6690 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:38:13.307611 6694 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 18:38:13.307799 6684 server_base.cc:1034] running on GCE node
I20260806 18:38:13.308375 6684 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 18:38:13.309369 6684 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 18:38:13.310554 6684 hybrid_clock.cc:648] HybridClock initialized: now 1786041493310481 us; error 86 us; skew 500 ppm
Aug 06 18:38:13 dist-test-slave-1xrh krb5kdc[6670](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041493, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.194.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 18:38:13.313764 6684 init.cc:377] Logged in from keytab as kudu/127.19.194.62@KRBTEST.COM (short username kudu)
I20260806 18:38:13.314723 6684 webserver.cc:460] Webserver started at http://127.19.194.62:38105/ using document root <none> and password file <none>
I20260806 18:38:13.315441 6684 fs_manager.cc:359] Metadata directory not provided
I20260806 18:38:13.315536 6684 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 18:38:13.315778 6684 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 18:38:13.318212 6684 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786041246094603-20232-0/minicluster-data/master-0/data/instance:
uuid: "3bdfdc815e254a5794b29843d6cf8bf1"
format_stamp: "Formatted at 2026-08-06 18:38:13 on dist-test-slave-1xrh"
server_key: "4267d30ce998a83811da33052a3ed949"
server_key_iv: "c57298adc86f753d4b2218f3f2d597ab"
server_key_version: "encryptionkey@0"
I20260806 18:38:13.318914 6684 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786041246094603-20232-0/minicluster-data/master-0/wal/instance:
uuid: "3bdfdc815e254a5794b29843d6cf8bf1"
format_stamp: "Formatted at 2026-08-06 18:38:13 on dist-test-slave-1xrh"
server_key: "4267d30ce998a83811da33052a3ed949"
server_key_iv: "c57298adc86f753d4b2218f3f2d597ab"
server_key_version: "encryptionkey@0"
I20260806 18:38:13.322667 6684 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.002s sys 0.000s
I20260806 18:38:13.325052 6702 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 18:38:13.326210 6684 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260806 18:38:13.326319 6684 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786041246094603-20232-0/minicluster-data/master-0/data,/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786041246094603-20232-0/minicluster-data/master-0/wal
uuid: "3bdfdc815e254a5794b29843d6cf8bf1"
format_stamp: "Formatted at 2026-08-06 18:38:13 on dist-test-slave-1xrh"
server_key: "4267d30ce998a83811da33052a3ed949"
server_key_iv: "c57298adc86f753d4b2218f3f2d597ab"
server_key_version: "encryptionkey@0"
I20260806 18:38:13.326421 6684 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786041246094603-20232-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786041246094603-20232-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786041246094603-20232-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 18:38:13.338800 6684 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 18:38:13.341535 6684 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 18:38:13.341681 6684 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 18:38:13.348726 6684 rpc_server.cc:307] RPC server started. Bound to: 127.19.194.62:40041
I20260806 18:38:13.348738 6755 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.194.62:40041 every 8 connection(s)
I20260806 18:38:13.349613 6684 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786041246094603-20232-0/minicluster-data/master-0/data/info.pb
I20260806 18:38:13.352270 6756 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 18:38:13.356556 20232 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJW69tn/build/debug/bin/kudu as pid 6684
I20260806 18:38:13.356689 20232 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786041246094603-20232-0/minicluster-data/master-0/wal/instance
I20260806 18:38:13.356999 20232 external_mini_cluster.cc:1428] Setting key 684df926c3b282123bf0192f0014f363
I20260806 18:38:13.358047 6756 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3bdfdc815e254a5794b29843d6cf8bf1: Bootstrap starting.
I20260806 18:38:13.360371 6756 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 3bdfdc815e254a5794b29843d6cf8bf1: Neither blocks nor log segments found. Creating new log.
I20260806 18:38:13.361239 6756 log.cc:824] T 00000000000000000000000000000000 P 3bdfdc815e254a5794b29843d6cf8bf1: Log is configured to *not* fsync() on all Append() calls
I20260806 18:38:13.363303 6756 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3bdfdc815e254a5794b29843d6cf8bf1: No bootstrap required, opened a new log
Aug 06 18:38:13 dist-test-slave-1xrh krb5kdc[6670](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041493, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.19.194.62@KRBTEST.COM
I20260806 18:38:13.367579 6756 raft_consensus.cc:348] T 00000000000000000000000000000000 P 3bdfdc815e254a5794b29843d6cf8bf1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3bdfdc815e254a5794b29843d6cf8bf1" member_type: VOTER last_known_addr { host: "127.19.194.62" port: 40041 } }
I20260806 18:38:13.367838 6756 raft_consensus.cc:374] T 00000000000000000000000000000000 P 3bdfdc815e254a5794b29843d6cf8bf1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 18:38:13.367935 6756 raft_consensus.cc:729] T 00000000000000000000000000000000 P 3bdfdc815e254a5794b29843d6cf8bf1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3bdfdc815e254a5794b29843d6cf8bf1, State: Initialized, Role: FOLLOWER
I20260806 18:38:13.368398 6756 consensus_queue.cc:260] T 00000000000000000000000000000000 P 3bdfdc815e254a5794b29843d6cf8bf1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3bdfdc815e254a5794b29843d6cf8bf1" member_type: VOTER last_known_addr { host: "127.19.194.62" port: 40041 } }
I20260806 18:38:13.368517 6756 raft_consensus.cc:388] T 00000000000000000000000000000000 P 3bdfdc815e254a5794b29843d6cf8bf1 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 18:38:13.368598 6756 raft_consensus.cc:482] T 00000000000000000000000000000000 P 3bdfdc815e254a5794b29843d6cf8bf1 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 18:38:13.368717 6756 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 3bdfdc815e254a5794b29843d6cf8bf1 [term 0 FOLLOWER]: Advancing to term 1
I20260806 18:38:13.369801 6756 raft_consensus.cc:504] T 00000000000000000000000000000000 P 3bdfdc815e254a5794b29843d6cf8bf1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3bdfdc815e254a5794b29843d6cf8bf1" member_type: VOTER last_known_addr { host: "127.19.194.62" port: 40041 } }
I20260806 18:38:13.370257 6756 leader_election.cc:302] T 00000000000000000000000000000000 P 3bdfdc815e254a5794b29843d6cf8bf1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 3bdfdc815e254a5794b29843d6cf8bf1; no voters:
I20260806 18:38:13.370623 6756 leader_election.cc:288] T 00000000000000000000000000000000 P 3bdfdc815e254a5794b29843d6cf8bf1 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 18:38:13.370779 6761 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 3bdfdc815e254a5794b29843d6cf8bf1 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 18:38:13.371047 6761 raft_consensus.cc:686] T 00000000000000000000000000000000 P 3bdfdc815e254a5794b29843d6cf8bf1 [term 1 LEADER]: Becoming Leader. State: Replica: 3bdfdc815e254a5794b29843d6cf8bf1, State: Running, Role: LEADER
I20260806 18:38:13.371428 6761 consensus_queue.cc:237] T 00000000000000000000000000000000 P 3bdfdc815e254a5794b29843d6cf8bf1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3bdfdc815e254a5794b29843d6cf8bf1" member_type: VOTER last_known_addr { host: "127.19.194.62" port: 40041 } }
I20260806 18:38:13.371923 6759 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:38:13.358383 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37984 (local address 127.19.194.62:40041)
0806 18:38:13.358786 (+ 403us) server_negotiation.cc:206] Beginning negotiation
0806 18:38:13.358802 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0806 18:38:13.358836 (+ 34us) server_negotiation.cc:407] Connection header received
0806 18:38:13.359536 (+ 700us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:38:13.359555 (+ 19us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:38:13.359861 (+ 306us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:38:13.360126 (+ 265us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:38:13.361095 (+ 969us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:38:13.362114 (+ 1019us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:38:13.363273 (+ 1159us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:38:13.363794 (+ 521us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:38:13.363922 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:38:13.366681 (+ 2759us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:38:13.366687 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:38:13.366700 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 18:38:13.366725 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:38:13.368613 (+ 1888us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:38:13.369115 (+ 502us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:38:13.369120 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:38:13.369125 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:38:13.369253 (+ 128us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:38:13.369546 (+ 293us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:38:13.369548 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:38:13.369549 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:38:13.370017 (+ 468us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:38:13.370168 (+ 151us) server_negotiation.cc:1035] Waiting for connection context
0806 18:38:13.370596 (+ 428us) server_negotiation.cc:299] Negotiation successful
0806 18:38:13.370913 (+ 317us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":256,"thread_start_us":139,"threads_started":1}
I20260806 18:38:13.372108 6756 sys_catalog.cc:564] T 00000000000000000000000000000000 P 3bdfdc815e254a5794b29843d6cf8bf1 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 18:38:13.374416 6762 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3bdfdc815e254a5794b29843d6cf8bf1 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "3bdfdc815e254a5794b29843d6cf8bf1" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3bdfdc815e254a5794b29843d6cf8bf1" member_type: VOTER last_known_addr { host: "127.19.194.62" port: 40041 } } }
I20260806 18:38:13.374547 6762 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3bdfdc815e254a5794b29843d6cf8bf1 [sys.catalog]: This master's current role is: LEADER
I20260806 18:38:13.374879 6763 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3bdfdc815e254a5794b29843d6cf8bf1 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 3bdfdc815e254a5794b29843d6cf8bf1. Latest consensus state: current_term: 1 leader_uuid: "3bdfdc815e254a5794b29843d6cf8bf1" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3bdfdc815e254a5794b29843d6cf8bf1" member_type: VOTER last_known_addr { host: "127.19.194.62" port: 40041 } } }
I20260806 18:38:13.375061 6763 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3bdfdc815e254a5794b29843d6cf8bf1 [sys.catalog]: This master's current role is: LEADER
W20260806 18:38:13.376647 6775 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 3bdfdc815e254a5794b29843d6cf8bf1: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260806 18:38:13.376713 6775 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260806 18:38:13.376792 6776 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 18:38:13.377526 6776 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 18:38:13.383069 6776 catalog_manager.cc:1347] Generated new cluster ID: 4d9b71c5fe67491796d9a180f60dc163
I20260806 18:38:13.383150 6776 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 18:38:13.388649 6776 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 18:38:13.389748 6776 catalog_manager.cc:1504] Loading token signing keys...
I20260806 18:38:13.408529 6776 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 3bdfdc815e254a5794b29843d6cf8bf1: Generated new TSK 0
I20260806 18:38:13.409268 6776 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.19.194.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.19.194.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.19.194.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786041246094603-20232-0/krb5kdc/kudu_127.19.194.1.keytab.
Entry for principal kudu/127.19.194.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786041246094603-20232-0/krb5kdc/kudu_127.19.194.1.keytab.
I20260806 18:38:13.442128 20232 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJW69tn/build/debug/bin/kudu
/tmp/dist-test-taskJW69tn/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786041246094603-20232-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786041246094603-20232-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786041246094603-20232-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786041246094603-20232-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.19.194.1:0
--local_ip_for_outbound_sockets=127.19.194.1
--webserver_interface=127.19.194.1
--webserver_port=0
--tserver_master_addrs=127.19.194.62:40041
--keytab_file=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786041246094603-20232-0/krb5kdc/kudu_127.19.194.1.keytab
--principal=kudu/127.19.194.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.19.194.20:37085
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786041246094603-20232-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786041246094603-20232-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786041246094603-20232-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 18:38:13.552167 6783 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 18:38:13.552448 6783 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 18:38:13.552525 6783 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 18:38:13.557410 6783 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 18:38:13.557497 6783 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 18:38:13.557590 6783 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.19.194.1
I20260806 18:38:13.562790 6783 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.194.20:37085
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786041246094603-20232-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786041246094603-20232-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786041246094603-20232-0/krb5kdc/kudu_127.19.194.1.keytab
--principal=kudu/127.19.194.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.19.194.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786041246094603-20232-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.19.194.1
--webserver_port=0
--tserver_master_addrs=127.19.194.62:40041
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6783
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.19.194.1
--log_dir=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786041246094603-20232-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 17:59:13 UTC on ccycloud.quasar-wvsnzx.root.comops.site
build id 4571028
I20260806 18:38:13.563896 6783 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 18:38:13.564877 6783 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 18:38:13.574499 6788 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:38:13.574465 6791 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:38:13.574478 6789 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 18:38:13.574586 6783 server_base.cc:1034] running on GCE node
I20260806 18:38:13.575074 6783 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 18:38:13.575767 6783 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 18:38:13.576954 6783 hybrid_clock.cc:648] HybridClock initialized: now 1786041493576929 us; error 46 us; skew 500 ppm
Aug 06 18:38:13 dist-test-slave-1xrh krb5kdc[6670](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041493, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.194.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 18:38:13.586426 6783 init.cc:377] Logged in from keytab as kudu/127.19.194.1@KRBTEST.COM (short username kudu)
I20260806 18:38:13.587342 6783 webserver.cc:460] Webserver started at http://127.19.194.1:35911/ using document root <none> and password file <none>
I20260806 18:38:13.587960 6783 fs_manager.cc:359] Metadata directory not provided
I20260806 18:38:13.588052 6783 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 18:38:13.588285 6783 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 18:38:13.590299 6783 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786041246094603-20232-0/minicluster-data/ts-0/data/instance:
uuid: "69be14cecf364c638c3881e7f9f90cbc"
format_stamp: "Formatted at 2026-08-06 18:38:13 on dist-test-slave-1xrh"
server_key: "ea5947a6f4511ff88d6a52c434546282"
server_key_iv: "5d81e26a0c2c16da29b533e1a2cac489"
server_key_version: "encryptionkey@0"
I20260806 18:38:13.590865 6783 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786041246094603-20232-0/minicluster-data/ts-0/wal/instance:
uuid: "69be14cecf364c638c3881e7f9f90cbc"
format_stamp: "Formatted at 2026-08-06 18:38:13 on dist-test-slave-1xrh"
server_key: "ea5947a6f4511ff88d6a52c434546282"
server_key_iv: "5d81e26a0c2c16da29b533e1a2cac489"
server_key_version: "encryptionkey@0"
I20260806 18:38:13.594439 6783 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.002s sys 0.000s
I20260806 18:38:13.596819 6799 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 18:38:13.598016 6783 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.000s sys 0.001s
I20260806 18:38:13.598142 6783 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786041246094603-20232-0/minicluster-data/ts-0/data,/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786041246094603-20232-0/minicluster-data/ts-0/wal
uuid: "69be14cecf364c638c3881e7f9f90cbc"
format_stamp: "Formatted at 2026-08-06 18:38:13 on dist-test-slave-1xrh"
server_key: "ea5947a6f4511ff88d6a52c434546282"
server_key_iv: "5d81e26a0c2c16da29b533e1a2cac489"
server_key_version: "encryptionkey@0"
I20260806 18:38:13.598249 6783 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786041246094603-20232-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786041246094603-20232-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786041246094603-20232-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 18:38:13.631883 6783 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 18:38:13.634847 6783 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 18:38:13.635082 6783 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 18:38:13.635730 6783 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 18:38:13.636577 6783 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 18:38:13.636651 6783 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 18:38:13.636725 6783 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 18:38:13.636777 6783 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 18:38:13.645958 6783 rpc_server.cc:307] RPC server started. Bound to: 127.19.194.1:37755
I20260806 18:38:13.645980 6920 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.194.1:37755 every 8 connection(s)
I20260806 18:38:13.646863 6783 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786041246094603-20232-0/minicluster-data/ts-0/data/info.pb
I20260806 18:38:13.649000 20232 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJW69tn/build/debug/bin/kudu as pid 6783
I20260806 18:38:13.649127 20232 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786041246094603-20232-0/minicluster-data/ts-0/wal/instance
I20260806 18:38:13.649434 20232 external_mini_cluster.cc:1428] Setting key c0736d8cde7b35d2a74078ee1e7e48a8
Aug 06 18:38:13 dist-test-slave-1xrh krb5kdc[6670](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041493, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.194.1@KRBTEST.COM for kudu/127.19.194.62@KRBTEST.COM
WARNING: no policy specified for kudu/127.19.194.2@KRBTEST.COM; defaulting to no policy
I20260806 18:38:13.662215 6759 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:38:13.648674 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.194.1:55221 (local address 127.19.194.62:40041)
0806 18:38:13.648866 (+ 192us) server_negotiation.cc:206] Beginning negotiation
0806 18:38:13.648873 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 18:38:13.649805 (+ 932us) server_negotiation.cc:407] Connection header received
0806 18:38:13.651274 (+ 1469us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:38:13.651276 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:38:13.651319 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:38:13.651382 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:38:13.653006 (+ 1624us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:38:13.653874 (+ 868us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:38:13.655003 (+ 1129us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:38:13.655487 (+ 484us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:38:13.655615 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:38:13.658406 (+ 2791us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:38:13.658412 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:38:13.658416 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 18:38:13.658459 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:38:13.660149 (+ 1690us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:38:13.660679 (+ 530us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:38:13.660681 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:38:13.660683 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:38:13.660758 (+ 75us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:38:13.661097 (+ 339us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:38:13.661099 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:38:13.661100 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:38:13.661272 (+ 172us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:38:13.661420 (+ 148us) server_negotiation.cc:1035] Waiting for connection context
0806 18:38:13.661979 (+ 559us) server_negotiation.cc:299] Negotiation successful
0806 18:38:13.662084 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.19.194.2@KRBTEST.COM" created.
I20260806 18:38:13.663213 6923 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:38:13.649059 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.194.62:40041 (local address 127.19.194.1:55221)
0806 18:38:13.649631 (+ 572us) negotiation.cc:107] Waiting for socket to connect
0806 18:38:13.649684 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0806 18:38:13.651027 (+ 1343us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:38:13.651574 (+ 547us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:38:13.651585 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:38:13.652127 (+ 542us) client_negotiation.cc:189] Negotiated authn=SASL
0806 18:38:13.652834 (+ 707us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:38:13.652856 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:38:13.654012 (+ 1156us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:38:13.654015 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:38:13.654874 (+ 859us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:38:13.654884 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:38:13.655669 (+ 785us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:38:13.655675 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:38:13.655856 (+ 181us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:38:13.656493 (+ 637us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 18:38:13.656510 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0806 18:38:13.658219 (+ 1709us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 18:38:13.660265 (+ 2046us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:38:13.660270 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:38:13.660305 (+ 35us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:38:13.660582 (+ 277us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:38:13.660844 (+ 262us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:38:13.660847 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:38:13.660848 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:38:13.661011 (+ 163us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:38:13.661424 (+ 413us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 18:38:13.661435 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 18:38:13.661719 (+ 284us) client_negotiation.cc:769] Sending connection context
0806 18:38:13.661934 (+ 215us) client_negotiation.cc:240] Negotiation successful
0806 18:38:13.662199 (+ 265us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":368,"thread_start_us":148,"threads_started":1}
I20260806 18:38:13.664541 6921 heartbeater.cc:347] Connected to a master server at 127.19.194.62:40041
I20260806 18:38:13.664796 6921 heartbeater.cc:464] Registering TS with master...
I20260806 18:38:13.665352 6921 heartbeater.cc:511] Master 127.19.194.62:40041 requested a full tablet report, sending...
I20260806 18:38:13.666921 6719 ts_manager.cc:194] Registered new tserver with Master: 69be14cecf364c638c3881e7f9f90cbc (127.19.194.1:37755)
I20260806 18:38:13.668066 6719 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.19.194.1@KRBTEST.COM'} at 127.19.194.1:55221
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.19.194.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786041246094603-20232-0/krb5kdc/kudu_127.19.194.2.keytab.
Entry for principal kudu/127.19.194.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786041246094603-20232-0/krb5kdc/kudu_127.19.194.2.keytab.
I20260806 18:38:13.680222 20232 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJW69tn/build/debug/bin/kudu
/tmp/dist-test-taskJW69tn/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786041246094603-20232-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786041246094603-20232-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786041246094603-20232-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786041246094603-20232-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.19.194.2:0
--local_ip_for_outbound_sockets=127.19.194.2
--webserver_interface=127.19.194.2
--webserver_port=0
--tserver_master_addrs=127.19.194.62:40041
--keytab_file=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786041246094603-20232-0/krb5kdc/kudu_127.19.194.2.keytab
--principal=kudu/127.19.194.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.19.194.20:37085
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786041246094603-20232-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786041246094603-20232-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786041246094603-20232-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 18:38:13.789222 6926 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 18:38:13.789450 6926 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 18:38:13.789490 6926 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 18:38:13.793269 6926 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 18:38:13.793310 6926 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 18:38:13.793361 6926 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.19.194.2
I20260806 18:38:13.799044 6926 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.194.20:37085
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786041246094603-20232-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786041246094603-20232-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786041246094603-20232-0/krb5kdc/kudu_127.19.194.2.keytab
--principal=kudu/127.19.194.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.19.194.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786041246094603-20232-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.19.194.2
--webserver_port=0
--tserver_master_addrs=127.19.194.62:40041
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6926
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.19.194.2
--log_dir=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786041246094603-20232-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 17:59:13 UTC on ccycloud.quasar-wvsnzx.root.comops.site
build id 4571028
I20260806 18:38:13.800423 6926 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 18:38:13.801579 6926 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 18:38:13.812662 6933 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:38:13.812639 6932 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:38:13.812633 6935 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 18:38:13.812985 6926 server_base.cc:1034] running on GCE node
I20260806 18:38:13.813405 6926 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 18:38:13.813951 6926 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 18:38:13.815178 6926 hybrid_clock.cc:648] HybridClock initialized: now 1786041493815164 us; error 27 us; skew 500 ppm
Aug 06 18:38:13 dist-test-slave-1xrh krb5kdc[6670](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041493, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.194.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 18:38:13.817744 6926 init.cc:377] Logged in from keytab as kudu/127.19.194.2@KRBTEST.COM (short username kudu)
I20260806 18:38:13.818570 6926 webserver.cc:460] Webserver started at http://127.19.194.2:46429/ using document root <none> and password file <none>
I20260806 18:38:13.819120 6926 fs_manager.cc:359] Metadata directory not provided
I20260806 18:38:13.819254 6926 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 18:38:13.819458 6926 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 18:38:13.821414 6926 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786041246094603-20232-0/minicluster-data/ts-1/data/instance:
uuid: "1d66bfcc0f164b148cc6c8be51f47a9a"
format_stamp: "Formatted at 2026-08-06 18:38:13 on dist-test-slave-1xrh"
server_key: "5758f8754c18e27d65879d89ba6c2342"
server_key_iv: "052217703f42eb83c57ef0629ab6f695"
server_key_version: "encryptionkey@0"
I20260806 18:38:13.821972 6926 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786041246094603-20232-0/minicluster-data/ts-1/wal/instance:
uuid: "1d66bfcc0f164b148cc6c8be51f47a9a"
format_stamp: "Formatted at 2026-08-06 18:38:13 on dist-test-slave-1xrh"
server_key: "5758f8754c18e27d65879d89ba6c2342"
server_key_iv: "052217703f42eb83c57ef0629ab6f695"
server_key_version: "encryptionkey@0"
I20260806 18:38:13.825748 6926 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.002s sys 0.003s
I20260806 18:38:13.828123 6942 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 18:38:13.829222 6926 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260806 18:38:13.829355 6926 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786041246094603-20232-0/minicluster-data/ts-1/data,/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786041246094603-20232-0/minicluster-data/ts-1/wal
uuid: "1d66bfcc0f164b148cc6c8be51f47a9a"
format_stamp: "Formatted at 2026-08-06 18:38:13 on dist-test-slave-1xrh"
server_key: "5758f8754c18e27d65879d89ba6c2342"
server_key_iv: "052217703f42eb83c57ef0629ab6f695"
server_key_version: "encryptionkey@0"
I20260806 18:38:13.829465 6926 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786041246094603-20232-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786041246094603-20232-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786041246094603-20232-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 18:38:13.846275 6926 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 18:38:13.848901 6926 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 18:38:13.849089 6926 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 18:38:13.849615 6926 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 18:38:13.850363 6926 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 18:38:13.850435 6926 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 18:38:13.850517 6926 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 18:38:13.850561 6926 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 18:38:13.859452 6926 rpc_server.cc:307] RPC server started. Bound to: 127.19.194.2:45107
I20260806 18:38:13.859483 7062 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.194.2:45107 every 8 connection(s)
I20260806 18:38:13.860224 6926 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786041246094603-20232-0/minicluster-data/ts-1/data/info.pb
I20260806 18:38:13.866555 20232 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJW69tn/build/debug/bin/kudu as pid 6926
I20260806 18:38:13.866694 20232 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786041246094603-20232-0/minicluster-data/ts-1/wal/instance
I20260806 18:38:13.866984 20232 external_mini_cluster.cc:1428] Setting key 7d72d25f6632c8574fadb7a390460968
Aug 06 18:38:13 dist-test-slave-1xrh krb5kdc[6670](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041493, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.194.2@KRBTEST.COM for kudu/127.19.194.62@KRBTEST.COM
I20260806 18:38:13.873955 6759 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:38:13.861922 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.194.2:56699 (local address 127.19.194.62:40041)
0806 18:38:13.862078 (+ 156us) server_negotiation.cc:206] Beginning negotiation
0806 18:38:13.862083 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 18:38:13.862877 (+ 794us) server_negotiation.cc:407] Connection header received
0806 18:38:13.864228 (+ 1351us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:38:13.864231 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:38:13.864278 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:38:13.864350 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:38:13.865398 (+ 1048us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:38:13.865905 (+ 507us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:38:13.867008 (+ 1103us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:38:13.867313 (+ 305us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:38:13.867426 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:38:13.869854 (+ 2428us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:38:13.869858 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:38:13.869861 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 18:38:13.869890 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:38:13.871377 (+ 1487us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:38:13.872204 (+ 827us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:38:13.872211 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:38:13.872215 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:38:13.872318 (+ 103us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:38:13.872686 (+ 368us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:38:13.872692 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:38:13.872697 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:38:13.873006 (+ 309us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:38:13.873119 (+ 113us) server_negotiation.cc:1035] Waiting for connection context
0806 18:38:13.873627 (+ 508us) server_negotiation.cc:299] Negotiation successful
0806 18:38:13.873762 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260806 18:38:13.874631 7066 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:38:13.862318 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.194.62:40041 (local address 127.19.194.2:56699)
0806 18:38:13.862734 (+ 416us) negotiation.cc:107] Waiting for socket to connect
0806 18:38:13.862774 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0806 18:38:13.864043 (+ 1269us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:38:13.864515 (+ 472us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:38:13.864520 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:38:13.864838 (+ 318us) client_negotiation.cc:189] Negotiated authn=SASL
0806 18:38:13.865270 (+ 432us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:38:13.865283 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:38:13.866011 (+ 728us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:38:13.866013 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:38:13.866871 (+ 858us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:38:13.866880 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:38:13.867482 (+ 602us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:38:13.867486 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:38:13.867677 (+ 191us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:38:13.868285 (+ 608us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 18:38:13.868305 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0806 18:38:13.869697 (+ 1392us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 18:38:13.871499 (+ 1802us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:38:13.871504 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:38:13.871518 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:38:13.872072 (+ 554us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:38:13.872434 (+ 362us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:38:13.872437 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:38:13.872438 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:38:13.872580 (+ 142us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:38:13.873119 (+ 539us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 18:38:13.873126 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 18:38:13.873390 (+ 264us) client_negotiation.cc:769] Sending connection context
0806 18:38:13.873549 (+ 159us) client_negotiation.cc:240] Negotiation successful
0806 18:38:13.873778 (+ 229us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":231,"thread_start_us":98,"threads_started":1}
I20260806 18:38:13.875840 7063 heartbeater.cc:347] Connected to a master server at 127.19.194.62:40041
I20260806 18:38:13.876060 7063 heartbeater.cc:464] Registering TS with master...
I20260806 18:38:13.876449 7063 heartbeater.cc:511] Master 127.19.194.62:40041 requested a full tablet report, sending...
I20260806 18:38:13.877411 6719 ts_manager.cc:194] Registered new tserver with Master: 1d66bfcc0f164b148cc6c8be51f47a9a (127.19.194.2:45107)
WARNING: no policy specified for kudu/127.19.194.3@KRBTEST.COM; defaulting to no policy
I20260806 18:38:13.877910 6719 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.19.194.2@KRBTEST.COM'} at 127.19.194.2:56699
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.19.194.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.19.194.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786041246094603-20232-0/krb5kdc/kudu_127.19.194.3.keytab.
Entry for principal kudu/127.19.194.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786041246094603-20232-0/krb5kdc/kudu_127.19.194.3.keytab.
I20260806 18:38:13.895862 20232 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJW69tn/build/debug/bin/kudu
/tmp/dist-test-taskJW69tn/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786041246094603-20232-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786041246094603-20232-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786041246094603-20232-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786041246094603-20232-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.19.194.3:0
--local_ip_for_outbound_sockets=127.19.194.3
--webserver_interface=127.19.194.3
--webserver_port=0
--tserver_master_addrs=127.19.194.62:40041
--keytab_file=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786041246094603-20232-0/krb5kdc/kudu_127.19.194.3.keytab
--principal=kudu/127.19.194.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.19.194.20:37085
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786041246094603-20232-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786041246094603-20232-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786041246094603-20232-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 18:38:14.005352 7069 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 18:38:14.005590 7069 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 18:38:14.005657 7069 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 18:38:14.010047 7069 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 18:38:14.010097 7069 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 18:38:14.010186 7069 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.19.194.3
I20260806 18:38:14.015413 7069 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.194.20:37085
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786041246094603-20232-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786041246094603-20232-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786041246094603-20232-0/krb5kdc/kudu_127.19.194.3.keytab
--principal=kudu/127.19.194.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.19.194.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786041246094603-20232-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.19.194.3
--webserver_port=0
--tserver_master_addrs=127.19.194.62:40041
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7069
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.19.194.3
--log_dir=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786041246094603-20232-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 17:59:13 UTC on ccycloud.quasar-wvsnzx.root.comops.site
build id 4571028
I20260806 18:38:14.016335 7069 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 18:38:14.017282 7069 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 18:38:14.026499 7076 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:38:14.026561 7075 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:38:14.026499 7078 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 18:38:14.026875 7069 server_base.cc:1034] running on GCE node
I20260806 18:38:14.027343 7069 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 18:38:14.027983 7069 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 18:38:14.029201 7069 hybrid_clock.cc:648] HybridClock initialized: now 1786041494029186 us; error 30 us; skew 500 ppm
Aug 06 18:38:14 dist-test-slave-1xrh krb5kdc[6670](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041494, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.194.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 18:38:14.032469 7069 init.cc:377] Logged in from keytab as kudu/127.19.194.3@KRBTEST.COM (short username kudu)
I20260806 18:38:14.033393 7069 webserver.cc:460] Webserver started at http://127.19.194.3:45633/ using document root <none> and password file <none>
I20260806 18:38:14.033972 7069 fs_manager.cc:359] Metadata directory not provided
I20260806 18:38:14.034065 7069 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 18:38:14.034291 7069 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 18:38:14.036441 7069 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786041246094603-20232-0/minicluster-data/ts-2/data/instance:
uuid: "a598fd9d7cf94ccc9f339d2560f35ab8"
format_stamp: "Formatted at 2026-08-06 18:38:14 on dist-test-slave-1xrh"
server_key: "43ca38ed5954087bf9dffdedbf79c30e"
server_key_iv: "e4d3a6a332cc9ba8ad488da9b6ae94e9"
server_key_version: "encryptionkey@0"
I20260806 18:38:14.037031 7069 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786041246094603-20232-0/minicluster-data/ts-2/wal/instance:
uuid: "a598fd9d7cf94ccc9f339d2560f35ab8"
format_stamp: "Formatted at 2026-08-06 18:38:14 on dist-test-slave-1xrh"
server_key: "43ca38ed5954087bf9dffdedbf79c30e"
server_key_iv: "e4d3a6a332cc9ba8ad488da9b6ae94e9"
server_key_version: "encryptionkey@0"
I20260806 18:38:14.040757 7069 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.005s sys 0.000s
I20260806 18:38:14.043316 7085 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 18:38:14.044528 7069 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260806 18:38:14.044667 7069 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786041246094603-20232-0/minicluster-data/ts-2/data,/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786041246094603-20232-0/minicluster-data/ts-2/wal
uuid: "a598fd9d7cf94ccc9f339d2560f35ab8"
format_stamp: "Formatted at 2026-08-06 18:38:14 on dist-test-slave-1xrh"
server_key: "43ca38ed5954087bf9dffdedbf79c30e"
server_key_iv: "e4d3a6a332cc9ba8ad488da9b6ae94e9"
server_key_version: "encryptionkey@0"
I20260806 18:38:14.044776 7069 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786041246094603-20232-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786041246094603-20232-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786041246094603-20232-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 18:38:14.064882 7069 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 18:38:14.067700 7069 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 18:38:14.067910 7069 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 18:38:14.068502 7069 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 18:38:14.069352 7069 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 18:38:14.069423 7069 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 18:38:14.069499 7069 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 18:38:14.069540 7069 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 18:38:14.078738 7069 rpc_server.cc:307] RPC server started. Bound to: 127.19.194.3:37707
I20260806 18:38:14.078759 7205 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.194.3:37707 every 8 connection(s)
I20260806 18:38:14.079648 7069 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786041246094603-20232-0/minicluster-data/ts-2/data/info.pb
I20260806 18:38:14.081735 20232 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJW69tn/build/debug/bin/kudu as pid 7069
I20260806 18:38:14.081872 20232 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786041246094603-20232-0/minicluster-data/ts-2/wal/instance
I20260806 18:38:14.082229 20232 external_mini_cluster.cc:1428] Setting key 69e012c7737e2251d3f5d7c79553e924
Aug 06 18:38:14 dist-test-slave-1xrh krb5kdc[6670](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041494, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.194.3@KRBTEST.COM for kudu/127.19.194.62@KRBTEST.COM
I20260806 18:38:14.093228 6759 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:38:14.081418 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.194.3:38641 (local address 127.19.194.62:40041)
0806 18:38:14.081586 (+ 168us) server_negotiation.cc:206] Beginning negotiation
0806 18:38:14.081591 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 18:38:14.082637 (+ 1046us) server_negotiation.cc:407] Connection header received
0806 18:38:14.084377 (+ 1740us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:38:14.084381 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:38:14.084438 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:38:14.084526 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:38:14.085630 (+ 1104us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:38:14.086169 (+ 539us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:38:14.087329 (+ 1160us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:38:14.087553 (+ 224us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:38:14.087634 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:38:14.090005 (+ 2371us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:38:14.090008 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:38:14.090010 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 18:38:14.090032 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:38:14.091413 (+ 1381us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:38:14.091906 (+ 493us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:38:14.091909 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:38:14.091911 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:38:14.091991 (+ 80us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:38:14.092311 (+ 320us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:38:14.092313 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:38:14.092314 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:38:14.092493 (+ 179us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:38:14.092570 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0806 18:38:14.093015 (+ 445us) server_negotiation.cc:299] Negotiation successful
0806 18:38:14.093109 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260806 18:38:14.094049 7208 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:38:14.081932 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.194.62:40041 (local address 127.19.194.3:38641)
0806 18:38:14.082459 (+ 527us) negotiation.cc:107] Waiting for socket to connect
0806 18:38:14.082517 (+ 58us) client_negotiation.cc:174] Beginning negotiation
0806 18:38:14.084131 (+ 1614us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:38:14.084686 (+ 555us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:38:14.084692 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:38:14.085034 (+ 342us) client_negotiation.cc:189] Negotiated authn=SASL
0806 18:38:14.085493 (+ 459us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:38:14.085506 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:38:14.086287 (+ 781us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:38:14.086289 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:38:14.087206 (+ 917us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:38:14.087217 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:38:14.087678 (+ 461us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:38:14.087681 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:38:14.087813 (+ 132us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:38:14.088426 (+ 613us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 18:38:14.088441 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0806 18:38:14.089849 (+ 1408us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 18:38:14.091515 (+ 1666us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:38:14.091520 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:38:14.091531 (+ 11us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:38:14.091771 (+ 240us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:38:14.092083 (+ 312us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:38:14.092085 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:38:14.092087 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:38:14.092225 (+ 138us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:38:14.092583 (+ 358us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 18:38:14.092588 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 18:38:14.092794 (+ 206us) client_negotiation.cc:769] Sending connection context
0806 18:38:14.092960 (+ 166us) client_negotiation.cc:240] Negotiation successful
0806 18:38:14.093191 (+ 231us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":279,"thread_start_us":106,"threads_started":1}
I20260806 18:38:14.095249 7206 heartbeater.cc:347] Connected to a master server at 127.19.194.62:40041
I20260806 18:38:14.095512 7206 heartbeater.cc:464] Registering TS with master...
I20260806 18:38:14.096083 7206 heartbeater.cc:511] Master 127.19.194.62:40041 requested a full tablet report, sending...
I20260806 18:38:14.097137 6719 ts_manager.cc:194] Registered new tserver with Master: a598fd9d7cf94ccc9f339d2560f35ab8 (127.19.194.3:37707)
I20260806 18:38:14.097697 6719 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.19.194.3@KRBTEST.COM'} at 127.19.194.3:38641
I20260806 18:38:14.108047 20232 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260806 18:38:14.117522 6759 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:38:14.109971 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37990 (local address 127.19.194.62:40041)
0806 18:38:14.110114 (+ 143us) server_negotiation.cc:206] Beginning negotiation
0806 18:38:14.110120 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 18:38:14.110358 (+ 238us) server_negotiation.cc:407] Connection header received
0806 18:38:14.110414 (+ 56us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:38:14.110416 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:38:14.110459 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:38:14.110522 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:38:14.111250 (+ 728us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:38:14.111746 (+ 496us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:38:14.112518 (+ 772us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:38:14.112715 (+ 197us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:38:14.112787 (+ 72us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:38:14.114249 (+ 1462us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:38:14.114252 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:38:14.114254 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 18:38:14.114276 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:38:14.115694 (+ 1418us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:38:14.116138 (+ 444us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:38:14.116140 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:38:14.116141 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:38:14.116210 (+ 69us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:38:14.116486 (+ 276us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:38:14.116488 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:38:14.116489 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:38:14.116668 (+ 179us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:38:14.116792 (+ 124us) server_negotiation.cc:1035] Waiting for connection context
0806 18:38:14.117039 (+ 247us) server_negotiation.cc:299] Negotiation successful
0806 18:38:14.117359 (+ 320us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260806 18:38:14.120639 6719 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:37990:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260806 18:38:14.123708 6719 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 18:38:14.138276 7222 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:38:14.132860 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37330 (local address 127.19.194.1:37755)
0806 18:38:14.133301 (+ 441us) server_negotiation.cc:206] Beginning negotiation
0806 18:38:14.133316 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0806 18:38:14.133739 (+ 423us) server_negotiation.cc:407] Connection header received
0806 18:38:14.133815 (+ 76us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:38:14.133820 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:38:14.134012 (+ 192us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:38:14.134146 (+ 134us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 18:38:14.134532 (+ 386us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:38:14.135332 (+ 800us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:38:14.136978 (+ 1646us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:38:14.137550 (+ 572us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:38:14.137693 (+ 143us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:38:14.137839 (+ 146us) server_negotiation.cc:1035] Waiting for connection context
0806 18:38:14.138021 (+ 182us) server_negotiation.cc:299] Negotiation successful
0806 18:38:14.138124 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":281,"thread_start_us":97,"threads_started":1}
I20260806 18:38:14.138267 7220 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:38:14.132698 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.194.1:37755 (local address 127.0.0.1:37330)
0806 18:38:14.133536 (+ 838us) negotiation.cc:107] Waiting for socket to connect
0806 18:38:14.133549 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0806 18:38:14.133633 (+ 84us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:38:14.134297 (+ 664us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:38:14.134299 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:38:14.134313 (+ 14us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 18:38:14.134417 (+ 104us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:38:14.134429 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:38:14.135467 (+ 1038us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:38:14.135473 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:38:14.136836 (+ 1363us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:38:14.136846 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:38:14.137691 (+ 845us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:38:14.137695 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:38:14.137796 (+ 101us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:38:14.137854 (+ 58us) client_negotiation.cc:769] Sending connection context
0806 18:38:14.137987 (+ 133us) client_negotiation.cc:240] Negotiation successful
0806 18:38:14.138124 (+ 137us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":781,"mutex_wait_us":28,"thread_start_us":88,"threads_started":1}
I20260806 18:38:14.138644 7218 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:38:14.132166 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42424 (local address 127.19.194.3:37707)
0806 18:38:14.133399 (+ 1233us) server_negotiation.cc:206] Beginning negotiation
0806 18:38:14.133413 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0806 18:38:14.133439 (+ 26us) server_negotiation.cc:407] Connection header received
0806 18:38:14.133515 (+ 76us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:38:14.133520 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:38:14.133810 (+ 290us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:38:14.134017 (+ 207us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 18:38:14.134426 (+ 409us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:38:14.135511 (+ 1085us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:38:14.137078 (+ 1567us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:38:14.137853 (+ 775us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:38:14.137954 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:38:14.138072 (+ 118us) server_negotiation.cc:1035] Waiting for connection context
0806 18:38:14.138394 (+ 322us) server_negotiation.cc:299] Negotiation successful
0806 18:38:14.138509 (+ 115us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1124,"thread_start_us":175,"threads_started":1}
I20260806 18:38:14.138684 7217 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:38:14.131969 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.194.3:37707 (local address 127.0.0.1:42424)
0806 18:38:14.133075 (+ 1106us) negotiation.cc:107] Waiting for socket to connect
0806 18:38:14.133125 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0806 18:38:14.133250 (+ 125us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:38:14.134191 (+ 941us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:38:14.134195 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:38:14.134212 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 18:38:14.134303 (+ 91us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:38:14.134309 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:38:14.135841 (+ 1532us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:38:14.135844 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:38:14.136966 (+ 1122us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:38:14.136974 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:38:14.138133 (+ 1159us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:38:14.138136 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:38:14.138216 (+ 80us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:38:14.138245 (+ 29us) client_negotiation.cc:769] Sending connection context
0806 18:38:14.138304 (+ 59us) client_negotiation.cc:240] Negotiation successful
0806 18:38:14.138552 (+ 248us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":945,"mutex_wait_us":207,"thread_start_us":88,"threads_started":1}
I20260806 18:38:14.139029 7219 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:38:14.132393 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.194.2:45107 (local address 127.0.0.1:37982)
0806 18:38:14.133389 (+ 996us) negotiation.cc:107] Waiting for socket to connect
0806 18:38:14.133402 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0806 18:38:14.133539 (+ 137us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:38:14.134001 (+ 462us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:38:14.134007 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:38:14.134029 (+ 22us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 18:38:14.134188 (+ 159us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:38:14.134198 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:38:14.135985 (+ 1787us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:38:14.135989 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:38:14.136775 (+ 786us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:38:14.136785 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:38:14.138064 (+ 1279us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:38:14.138068 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:38:14.138178 (+ 110us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:38:14.138204 (+ 26us) client_negotiation.cc:769] Sending connection context
0806 18:38:14.138275 (+ 71us) client_negotiation.cc:240] Negotiation successful
0806 18:38:14.138500 (+ 225us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":927,"thread_start_us":93,"threads_started":1}
I20260806 18:38:14.139578 7221 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:38:14.132835 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37982 (local address 127.19.194.2:45107)
0806 18:38:14.133240 (+ 405us) server_negotiation.cc:206] Beginning negotiation
0806 18:38:14.133254 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0806 18:38:14.133512 (+ 258us) server_negotiation.cc:407] Connection header received
0806 18:38:14.133678 (+ 166us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:38:14.133683 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:38:14.133832 (+ 149us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:38:14.134009 (+ 177us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 18:38:14.134293 (+ 284us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:38:14.135843 (+ 1550us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:38:14.136907 (+ 1064us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:38:14.137722 (+ 815us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:38:14.137834 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:38:14.137996 (+ 162us) server_negotiation.cc:1035] Waiting for connection context
0806 18:38:14.139333 (+ 1337us) server_negotiation.cc:299] Negotiation successful
0806 18:38:14.139434 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":318,"thread_start_us":107,"threads_started":1}
I20260806 18:38:14.141722 7138 tablet_service.cc:1467] Processing CreateTablet for tablet 3203988bbf1942fbacc78dc10f0fb48e (DEFAULT_TABLE table=test-table [id=68982399de934429bfef89b9a8a1b479]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 18:38:14.141641 6852 tablet_service.cc:1467] Processing CreateTablet for tablet 3203988bbf1942fbacc78dc10f0fb48e (DEFAULT_TABLE table=test-table [id=68982399de934429bfef89b9a8a1b479]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 18:38:14.142069 6995 tablet_service.cc:1467] Processing CreateTablet for tablet 3203988bbf1942fbacc78dc10f0fb48e (DEFAULT_TABLE table=test-table [id=68982399de934429bfef89b9a8a1b479]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 18:38:14.142714 6852 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3203988bbf1942fbacc78dc10f0fb48e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 18:38:14.142725 7138 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3203988bbf1942fbacc78dc10f0fb48e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 18:38:14.142944 6995 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3203988bbf1942fbacc78dc10f0fb48e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 18:38:14.149688 7223 tablet_bootstrap.cc:492] T 3203988bbf1942fbacc78dc10f0fb48e P a598fd9d7cf94ccc9f339d2560f35ab8: Bootstrap starting.
I20260806 18:38:14.150444 7225 tablet_bootstrap.cc:492] T 3203988bbf1942fbacc78dc10f0fb48e P 1d66bfcc0f164b148cc6c8be51f47a9a: Bootstrap starting.
I20260806 18:38:14.150446 7224 tablet_bootstrap.cc:492] T 3203988bbf1942fbacc78dc10f0fb48e P 69be14cecf364c638c3881e7f9f90cbc: Bootstrap starting.
I20260806 18:38:14.152150 7224 tablet_bootstrap.cc:654] T 3203988bbf1942fbacc78dc10f0fb48e P 69be14cecf364c638c3881e7f9f90cbc: Neither blocks nor log segments found. Creating new log.
I20260806 18:38:14.152307 7223 tablet_bootstrap.cc:654] T 3203988bbf1942fbacc78dc10f0fb48e P a598fd9d7cf94ccc9f339d2560f35ab8: Neither blocks nor log segments found. Creating new log.
I20260806 18:38:14.152850 7225 tablet_bootstrap.cc:654] T 3203988bbf1942fbacc78dc10f0fb48e P 1d66bfcc0f164b148cc6c8be51f47a9a: Neither blocks nor log segments found. Creating new log.
I20260806 18:38:14.153422 7224 log.cc:824] T 3203988bbf1942fbacc78dc10f0fb48e P 69be14cecf364c638c3881e7f9f90cbc: Log is configured to *not* fsync() on all Append() calls
I20260806 18:38:14.153672 7225 log.cc:824] T 3203988bbf1942fbacc78dc10f0fb48e P 1d66bfcc0f164b148cc6c8be51f47a9a: Log is configured to *not* fsync() on all Append() calls
I20260806 18:38:14.153672 7223 log.cc:824] T 3203988bbf1942fbacc78dc10f0fb48e P a598fd9d7cf94ccc9f339d2560f35ab8: Log is configured to *not* fsync() on all Append() calls
I20260806 18:38:14.155328 7224 tablet_bootstrap.cc:492] T 3203988bbf1942fbacc78dc10f0fb48e P 69be14cecf364c638c3881e7f9f90cbc: No bootstrap required, opened a new log
I20260806 18:38:14.155537 7224 ts_tablet_manager.cc:1397] T 3203988bbf1942fbacc78dc10f0fb48e P 69be14cecf364c638c3881e7f9f90cbc: Time spent bootstrapping tablet: real 0.005s user 0.005s sys 0.000s
I20260806 18:38:14.155601 7225 tablet_bootstrap.cc:492] T 3203988bbf1942fbacc78dc10f0fb48e P 1d66bfcc0f164b148cc6c8be51f47a9a: No bootstrap required, opened a new log
I20260806 18:38:14.155606 7223 tablet_bootstrap.cc:492] T 3203988bbf1942fbacc78dc10f0fb48e P a598fd9d7cf94ccc9f339d2560f35ab8: No bootstrap required, opened a new log
I20260806 18:38:14.155781 7223 ts_tablet_manager.cc:1397] T 3203988bbf1942fbacc78dc10f0fb48e P a598fd9d7cf94ccc9f339d2560f35ab8: Time spent bootstrapping tablet: real 0.006s user 0.005s sys 0.000s
I20260806 18:38:14.155781 7225 ts_tablet_manager.cc:1397] T 3203988bbf1942fbacc78dc10f0fb48e P 1d66bfcc0f164b148cc6c8be51f47a9a: Time spent bootstrapping tablet: real 0.005s user 0.005s sys 0.000s
I20260806 18:38:14.158387 7224 raft_consensus.cc:348] T 3203988bbf1942fbacc78dc10f0fb48e P 69be14cecf364c638c3881e7f9f90cbc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a598fd9d7cf94ccc9f339d2560f35ab8" member_type: VOTER last_known_addr { host: "127.19.194.3" port: 37707 } } peers { permanent_uuid: "69be14cecf364c638c3881e7f9f90cbc" member_type: VOTER last_known_addr { host: "127.19.194.1" port: 37755 } } peers { permanent_uuid: "1d66bfcc0f164b148cc6c8be51f47a9a" member_type: VOTER last_known_addr { host: "127.19.194.2" port: 45107 } }
I20260806 18:38:14.158646 7224 raft_consensus.cc:374] T 3203988bbf1942fbacc78dc10f0fb48e P 69be14cecf364c638c3881e7f9f90cbc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 18:38:14.158708 7224 raft_consensus.cc:729] T 3203988bbf1942fbacc78dc10f0fb48e P 69be14cecf364c638c3881e7f9f90cbc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 69be14cecf364c638c3881e7f9f90cbc, State: Initialized, Role: FOLLOWER
I20260806 18:38:14.159194 7224 consensus_queue.cc:260] T 3203988bbf1942fbacc78dc10f0fb48e P 69be14cecf364c638c3881e7f9f90cbc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a598fd9d7cf94ccc9f339d2560f35ab8" member_type: VOTER last_known_addr { host: "127.19.194.3" port: 37707 } } peers { permanent_uuid: "69be14cecf364c638c3881e7f9f90cbc" member_type: VOTER last_known_addr { host: "127.19.194.1" port: 37755 } } peers { permanent_uuid: "1d66bfcc0f164b148cc6c8be51f47a9a" member_type: VOTER last_known_addr { host: "127.19.194.2" port: 45107 } }
I20260806 18:38:14.159538 7223 raft_consensus.cc:348] T 3203988bbf1942fbacc78dc10f0fb48e P a598fd9d7cf94ccc9f339d2560f35ab8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a598fd9d7cf94ccc9f339d2560f35ab8" member_type: VOTER last_known_addr { host: "127.19.194.3" port: 37707 } } peers { permanent_uuid: "69be14cecf364c638c3881e7f9f90cbc" member_type: VOTER last_known_addr { host: "127.19.194.1" port: 37755 } } peers { permanent_uuid: "1d66bfcc0f164b148cc6c8be51f47a9a" member_type: VOTER last_known_addr { host: "127.19.194.2" port: 45107 } }
I20260806 18:38:14.159782 7223 raft_consensus.cc:374] T 3203988bbf1942fbacc78dc10f0fb48e P a598fd9d7cf94ccc9f339d2560f35ab8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 18:38:14.159835 7223 raft_consensus.cc:729] T 3203988bbf1942fbacc78dc10f0fb48e P a598fd9d7cf94ccc9f339d2560f35ab8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a598fd9d7cf94ccc9f339d2560f35ab8, State: Initialized, Role: FOLLOWER
I20260806 18:38:14.159830 6921 heartbeater.cc:503] Master 127.19.194.62:40041 was elected leader, sending a full tablet report...
I20260806 18:38:14.160181 7224 ts_tablet_manager.cc:1428] T 3203988bbf1942fbacc78dc10f0fb48e P 69be14cecf364c638c3881e7f9f90cbc: Time spent starting tablet: real 0.005s user 0.004s sys 0.000s
I20260806 18:38:14.160239 7223 consensus_queue.cc:260] T 3203988bbf1942fbacc78dc10f0fb48e P a598fd9d7cf94ccc9f339d2560f35ab8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a598fd9d7cf94ccc9f339d2560f35ab8" member_type: VOTER last_known_addr { host: "127.19.194.3" port: 37707 } } peers { permanent_uuid: "69be14cecf364c638c3881e7f9f90cbc" member_type: VOTER last_known_addr { host: "127.19.194.1" port: 37755 } } peers { permanent_uuid: "1d66bfcc0f164b148cc6c8be51f47a9a" member_type: VOTER last_known_addr { host: "127.19.194.2" port: 45107 } }
I20260806 18:38:14.159538 7225 raft_consensus.cc:348] T 3203988bbf1942fbacc78dc10f0fb48e P 1d66bfcc0f164b148cc6c8be51f47a9a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a598fd9d7cf94ccc9f339d2560f35ab8" member_type: VOTER last_known_addr { host: "127.19.194.3" port: 37707 } } peers { permanent_uuid: "69be14cecf364c638c3881e7f9f90cbc" member_type: VOTER last_known_addr { host: "127.19.194.1" port: 37755 } } peers { permanent_uuid: "1d66bfcc0f164b148cc6c8be51f47a9a" member_type: VOTER last_known_addr { host: "127.19.194.2" port: 45107 } }
I20260806 18:38:14.160604 7225 raft_consensus.cc:374] T 3203988bbf1942fbacc78dc10f0fb48e P 1d66bfcc0f164b148cc6c8be51f47a9a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 18:38:14.160661 7225 raft_consensus.cc:729] T 3203988bbf1942fbacc78dc10f0fb48e P 1d66bfcc0f164b148cc6c8be51f47a9a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1d66bfcc0f164b148cc6c8be51f47a9a, State: Initialized, Role: FOLLOWER
I20260806 18:38:14.161036 7225 consensus_queue.cc:260] T 3203988bbf1942fbacc78dc10f0fb48e P 1d66bfcc0f164b148cc6c8be51f47a9a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a598fd9d7cf94ccc9f339d2560f35ab8" member_type: VOTER last_known_addr { host: "127.19.194.3" port: 37707 } } peers { permanent_uuid: "69be14cecf364c638c3881e7f9f90cbc" member_type: VOTER last_known_addr { host: "127.19.194.1" port: 37755 } } peers { permanent_uuid: "1d66bfcc0f164b148cc6c8be51f47a9a" member_type: VOTER last_known_addr { host: "127.19.194.2" port: 45107 } }
I20260806 18:38:14.161149 7223 ts_tablet_manager.cc:1428] T 3203988bbf1942fbacc78dc10f0fb48e P a598fd9d7cf94ccc9f339d2560f35ab8: Time spent starting tablet: real 0.005s user 0.005s sys 0.000s
I20260806 18:38:14.161234 7206 heartbeater.cc:503] Master 127.19.194.62:40041 was elected leader, sending a full tablet report...
I20260806 18:38:14.161772 7225 ts_tablet_manager.cc:1428] T 3203988bbf1942fbacc78dc10f0fb48e P 1d66bfcc0f164b148cc6c8be51f47a9a: Time spent starting tablet: real 0.006s user 0.005s sys 0.000s
I20260806 18:38:14.162758 7063 heartbeater.cc:503] Master 127.19.194.62:40041 was elected leader, sending a full tablet report...
I20260806 18:38:14.213352 7231 raft_consensus.cc:482] T 3203988bbf1942fbacc78dc10f0fb48e P 1d66bfcc0f164b148cc6c8be51f47a9a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 18:38:14.213531 7231 raft_consensus.cc:504] T 3203988bbf1942fbacc78dc10f0fb48e P 1d66bfcc0f164b148cc6c8be51f47a9a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a598fd9d7cf94ccc9f339d2560f35ab8" member_type: VOTER last_known_addr { host: "127.19.194.3" port: 37707 } } peers { permanent_uuid: "69be14cecf364c638c3881e7f9f90cbc" member_type: VOTER last_known_addr { host: "127.19.194.1" port: 37755 } } peers { permanent_uuid: "1d66bfcc0f164b148cc6c8be51f47a9a" member_type: VOTER last_known_addr { host: "127.19.194.2" port: 45107 } }
I20260806 18:38:14.214613 7231 leader_election.cc:288] T 3203988bbf1942fbacc78dc10f0fb48e P 1d66bfcc0f164b148cc6c8be51f47a9a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a598fd9d7cf94ccc9f339d2560f35ab8 (127.19.194.3:37707), 69be14cecf364c638c3881e7f9f90cbc (127.19.194.1:37755)
I20260806 18:38:14.218091 7218 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:38:14.214974 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.194.2:57447 (local address 127.19.194.3:37707)
0806 18:38:14.215110 (+ 136us) server_negotiation.cc:206] Beginning negotiation
0806 18:38:14.215115 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 18:38:14.215166 (+ 51us) server_negotiation.cc:407] Connection header received
0806 18:38:14.215223 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:38:14.215226 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:38:14.215274 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:38:14.215360 (+ 86us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 18:38:14.215638 (+ 278us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:38:14.216117 (+ 479us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:38:14.217141 (+ 1024us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:38:14.217552 (+ 411us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:38:14.217679 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:38:14.217736 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0806 18:38:14.217906 (+ 170us) server_negotiation.cc:299] Negotiation successful
0806 18:38:14.217976 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260806 18:38:14.218091 7066 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:38:14.214826 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.194.3:37707 (local address 127.19.194.2:57447)
0806 18:38:14.214980 (+ 154us) negotiation.cc:107] Waiting for socket to connect
0806 18:38:14.214997 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0806 18:38:14.215088 (+ 91us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:38:14.215376 (+ 288us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:38:14.215379 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:38:14.215398 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 18:38:14.215519 (+ 121us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:38:14.215525 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:38:14.216238 (+ 713us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:38:14.216242 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:38:14.216997 (+ 755us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:38:14.217005 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:38:14.217672 (+ 667us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:38:14.217675 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:38:14.217777 (+ 102us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:38:14.217804 (+ 27us) client_negotiation.cc:769] Sending connection context
0806 18:38:14.217889 (+ 85us) client_negotiation.cc:240] Negotiation successful
0806 18:38:14.217963 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":52}
I20260806 18:38:14.218614 7232 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:38:14.215115 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.194.1:37755 (local address 127.19.194.2:44835)
0806 18:38:14.215572 (+ 457us) negotiation.cc:107] Waiting for socket to connect
0806 18:38:14.215583 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0806 18:38:14.215693 (+ 110us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:38:14.215928 (+ 235us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:38:14.215930 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:38:14.215946 (+ 16us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 18:38:14.216076 (+ 130us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:38:14.216082 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:38:14.216770 (+ 688us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:38:14.216775 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:38:14.217560 (+ 785us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:38:14.217568 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:38:14.218211 (+ 643us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:38:14.218215 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:38:14.218339 (+ 124us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:38:14.218359 (+ 20us) client_negotiation.cc:769] Sending connection context
0806 18:38:14.218435 (+ 76us) client_negotiation.cc:240] Negotiation successful
0806 18:38:14.218501 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":406,"thread_start_us":142,"threads_started":1}
I20260806 18:38:14.218657 7222 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:38:14.215258 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.194.2:44835 (local address 127.19.194.1:37755)
0806 18:38:14.215413 (+ 155us) server_negotiation.cc:206] Beginning negotiation
0806 18:38:14.215417 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 18:38:14.215664 (+ 247us) server_negotiation.cc:407] Connection header received
0806 18:38:14.215787 (+ 123us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:38:14.215790 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:38:14.215829 (+ 39us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:38:14.215916 (+ 87us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 18:38:14.216196 (+ 280us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:38:14.216660 (+ 464us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:38:14.217689 (+ 1029us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:38:14.218073 (+ 384us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:38:14.218199 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:38:14.218247 (+ 48us) server_negotiation.cc:1035] Waiting for connection context
0806 18:38:14.218464 (+ 217us) server_negotiation.cc:299] Negotiation successful
0806 18:38:14.218547 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260806 18:38:14.218750 7158 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3203988bbf1942fbacc78dc10f0fb48e" candidate_uuid: "1d66bfcc0f164b148cc6c8be51f47a9a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a598fd9d7cf94ccc9f339d2560f35ab8" is_pre_election: true
I20260806 18:38:14.219064 7158 raft_consensus.cc:2454] T 3203988bbf1942fbacc78dc10f0fb48e P a598fd9d7cf94ccc9f339d2560f35ab8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1d66bfcc0f164b148cc6c8be51f47a9a in term 0.
I20260806 18:38:14.219353 6872 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3203988bbf1942fbacc78dc10f0fb48e" candidate_uuid: "1d66bfcc0f164b148cc6c8be51f47a9a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "69be14cecf364c638c3881e7f9f90cbc" is_pre_election: true
I20260806 18:38:14.219609 6947 leader_election.cc:302] T 3203988bbf1942fbacc78dc10f0fb48e P 1d66bfcc0f164b148cc6c8be51f47a9a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1d66bfcc0f164b148cc6c8be51f47a9a, a598fd9d7cf94ccc9f339d2560f35ab8; no voters:
I20260806 18:38:14.219666 6872 raft_consensus.cc:2454] T 3203988bbf1942fbacc78dc10f0fb48e P 69be14cecf364c638c3881e7f9f90cbc [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1d66bfcc0f164b148cc6c8be51f47a9a in term 0.
I20260806 18:38:14.219882 7231 raft_consensus.cc:2781] T 3203988bbf1942fbacc78dc10f0fb48e P 1d66bfcc0f164b148cc6c8be51f47a9a [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 18:38:14.220012 7231 raft_consensus.cc:482] T 3203988bbf1942fbacc78dc10f0fb48e P 1d66bfcc0f164b148cc6c8be51f47a9a [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 18:38:14.220072 7231 raft_consensus.cc:3037] T 3203988bbf1942fbacc78dc10f0fb48e P 1d66bfcc0f164b148cc6c8be51f47a9a [term 0 FOLLOWER]: Advancing to term 1
I20260806 18:38:14.221141 7231 raft_consensus.cc:504] T 3203988bbf1942fbacc78dc10f0fb48e P 1d66bfcc0f164b148cc6c8be51f47a9a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a598fd9d7cf94ccc9f339d2560f35ab8" member_type: VOTER last_known_addr { host: "127.19.194.3" port: 37707 } } peers { permanent_uuid: "69be14cecf364c638c3881e7f9f90cbc" member_type: VOTER last_known_addr { host: "127.19.194.1" port: 37755 } } peers { permanent_uuid: "1d66bfcc0f164b148cc6c8be51f47a9a" member_type: VOTER last_known_addr { host: "127.19.194.2" port: 45107 } }
I20260806 18:38:14.221475 7231 leader_election.cc:288] T 3203988bbf1942fbacc78dc10f0fb48e P 1d66bfcc0f164b148cc6c8be51f47a9a [CANDIDATE]: Term 1 election: Requested vote from peers a598fd9d7cf94ccc9f339d2560f35ab8 (127.19.194.3:37707), 69be14cecf364c638c3881e7f9f90cbc (127.19.194.1:37755)
I20260806 18:38:14.221844 7158 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3203988bbf1942fbacc78dc10f0fb48e" candidate_uuid: "1d66bfcc0f164b148cc6c8be51f47a9a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a598fd9d7cf94ccc9f339d2560f35ab8"
I20260806 18:38:14.221837 6872 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3203988bbf1942fbacc78dc10f0fb48e" candidate_uuid: "1d66bfcc0f164b148cc6c8be51f47a9a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "69be14cecf364c638c3881e7f9f90cbc"
I20260806 18:38:14.221988 7158 raft_consensus.cc:3037] T 3203988bbf1942fbacc78dc10f0fb48e P a598fd9d7cf94ccc9f339d2560f35ab8 [term 0 FOLLOWER]: Advancing to term 1
I20260806 18:38:14.222003 6872 raft_consensus.cc:3037] T 3203988bbf1942fbacc78dc10f0fb48e P 69be14cecf364c638c3881e7f9f90cbc [term 0 FOLLOWER]: Advancing to term 1
I20260806 18:38:14.223176 6872 raft_consensus.cc:2454] T 3203988bbf1942fbacc78dc10f0fb48e P 69be14cecf364c638c3881e7f9f90cbc [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1d66bfcc0f164b148cc6c8be51f47a9a in term 1.
I20260806 18:38:14.223282 7158 raft_consensus.cc:2454] T 3203988bbf1942fbacc78dc10f0fb48e P a598fd9d7cf94ccc9f339d2560f35ab8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1d66bfcc0f164b148cc6c8be51f47a9a in term 1.
I20260806 18:38:14.223529 6947 leader_election.cc:302] T 3203988bbf1942fbacc78dc10f0fb48e P 1d66bfcc0f164b148cc6c8be51f47a9a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1d66bfcc0f164b148cc6c8be51f47a9a, 69be14cecf364c638c3881e7f9f90cbc; no voters:
I20260806 18:38:14.223721 7231 raft_consensus.cc:2781] T 3203988bbf1942fbacc78dc10f0fb48e P 1d66bfcc0f164b148cc6c8be51f47a9a [term 1 FOLLOWER]: Leader election won for term 1
I20260806 18:38:14.223997 7231 raft_consensus.cc:686] T 3203988bbf1942fbacc78dc10f0fb48e P 1d66bfcc0f164b148cc6c8be51f47a9a [term 1 LEADER]: Becoming Leader. State: Replica: 1d66bfcc0f164b148cc6c8be51f47a9a, State: Running, Role: LEADER
I20260806 18:38:14.224316 7231 consensus_queue.cc:237] T 3203988bbf1942fbacc78dc10f0fb48e P 1d66bfcc0f164b148cc6c8be51f47a9a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a598fd9d7cf94ccc9f339d2560f35ab8" member_type: VOTER last_known_addr { host: "127.19.194.3" port: 37707 } } peers { permanent_uuid: "69be14cecf364c638c3881e7f9f90cbc" member_type: VOTER last_known_addr { host: "127.19.194.1" port: 37755 } } peers { permanent_uuid: "1d66bfcc0f164b148cc6c8be51f47a9a" member_type: VOTER last_known_addr { host: "127.19.194.2" port: 45107 } }
I20260806 18:38:14.227506 6717 catalog_manager.cc:5495] T 3203988bbf1942fbacc78dc10f0fb48e P 1d66bfcc0f164b148cc6c8be51f47a9a reported cstate change: term changed from 0 to 1, leader changed from <none> to 1d66bfcc0f164b148cc6c8be51f47a9a (127.19.194.2). New cstate: current_term: 1 leader_uuid: "1d66bfcc0f164b148cc6c8be51f47a9a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a598fd9d7cf94ccc9f339d2560f35ab8" member_type: VOTER last_known_addr { host: "127.19.194.3" port: 37707 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "69be14cecf364c638c3881e7f9f90cbc" member_type: VOTER last_known_addr { host: "127.19.194.1" port: 37755 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1d66bfcc0f164b148cc6c8be51f47a9a" member_type: VOTER last_known_addr { host: "127.19.194.2" port: 45107 } health_report { overall_health: HEALTHY } } }
I20260806 18:38:14.258692 7221 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:38:14.255518 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37996 (local address 127.19.194.2:45107)
0806 18:38:14.255654 (+ 136us) server_negotiation.cc:206] Beginning negotiation
0806 18:38:14.255657 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 18:38:14.255671 (+ 14us) server_negotiation.cc:407] Connection header received
0806 18:38:14.255807 (+ 136us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:38:14.255810 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:38:14.255862 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:38:14.255957 (+ 95us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 18:38:14.256175 (+ 218us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:38:14.256642 (+ 467us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:38:14.257420 (+ 778us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:38:14.257614 (+ 194us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:38:14.257684 (+ 70us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:38:14.257971 (+ 287us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 18:38:14.258345 (+ 374us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 18:38:14.258451 (+ 106us) server_negotiation.cc:1035] Waiting for connection context
0806 18:38:14.258557 (+ 106us) server_negotiation.cc:299] Negotiation successful
0806 18:38:14.258589 (+ 32us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260806 18:38:14.264199 6872 raft_consensus.cc:1261] T 3203988bbf1942fbacc78dc10f0fb48e P 69be14cecf364c638c3881e7f9f90cbc [term 1 FOLLOWER]: Refusing update from remote peer 1d66bfcc0f164b148cc6c8be51f47a9a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 18:38:14.264217 7158 raft_consensus.cc:1261] T 3203988bbf1942fbacc78dc10f0fb48e P a598fd9d7cf94ccc9f339d2560f35ab8 [term 1 FOLLOWER]: Refusing update from remote peer 1d66bfcc0f164b148cc6c8be51f47a9a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 18:38:14.264938 7233 consensus_queue.cc:1035] T 3203988bbf1942fbacc78dc10f0fb48e P 1d66bfcc0f164b148cc6c8be51f47a9a [LEADER]: Connected to new peer: Peer: permanent_uuid: "69be14cecf364c638c3881e7f9f90cbc" member_type: VOTER last_known_addr { host: "127.19.194.1" port: 37755 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 18:38:14.265195 7231 consensus_queue.cc:1035] T 3203988bbf1942fbacc78dc10f0fb48e P 1d66bfcc0f164b148cc6c8be51f47a9a [LEADER]: Connected to new peer: Peer: permanent_uuid: "a598fd9d7cf94ccc9f339d2560f35ab8" member_type: VOTER last_known_addr { host: "127.19.194.3" port: 37707 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 18:38:14.273725 7237 mvcc.cc:205] Tried to move back new op lower bound from 7315625960499097600 to 7315625960345460736. Current Snapshot: MvccSnapshot[applied={T|T < 7315625960499097600}]
I20260806 18:38:14.281098 6718 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:37990:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 18:38:14.281239 6718 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:37990:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 18:38:14.283596 6718 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 3bdfdc815e254a5794b29843d6cf8bf1: Sending DeleteTablet for 3 replicas of tablet 3203988bbf1942fbacc78dc10f0fb48e
I20260806 18:38:14.284384 6995 tablet_service.cc:1514] Processing DeleteTablet for tablet 3203988bbf1942fbacc78dc10f0fb48e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 18:38:14 UTC) from {username='kudu', principal='kudu/127.19.194.62@KRBTEST.COM'} at 127.0.0.1:37982
I20260806 18:38:14.284605 6852 tablet_service.cc:1514] Processing DeleteTablet for tablet 3203988bbf1942fbacc78dc10f0fb48e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 18:38:14 UTC) from {username='kudu', principal='kudu/127.19.194.62@KRBTEST.COM'} at 127.0.0.1:37330
I20260806 18:38:14.284943 7246 tablet_replica.cc:330] T 3203988bbf1942fbacc78dc10f0fb48e P 1d66bfcc0f164b148cc6c8be51f47a9a: stopping tablet replica
I20260806 18:38:14.284883 7138 tablet_service.cc:1514] Processing DeleteTablet for tablet 3203988bbf1942fbacc78dc10f0fb48e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 18:38:14 UTC) from {username='kudu', principal='kudu/127.19.194.62@KRBTEST.COM'} at 127.0.0.1:42424
I20260806 18:38:14.285037 7247 tablet_replica.cc:330] T 3203988bbf1942fbacc78dc10f0fb48e P 69be14cecf364c638c3881e7f9f90cbc: stopping tablet replica
I20260806 18:38:14.285336 7246 raft_consensus.cc:2229] T 3203988bbf1942fbacc78dc10f0fb48e P 1d66bfcc0f164b148cc6c8be51f47a9a [term 1 LEADER]: Raft consensus shutting down.
I20260806 18:38:14.285339 7247 raft_consensus.cc:2229] T 3203988bbf1942fbacc78dc10f0fb48e P 69be14cecf364c638c3881e7f9f90cbc [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 18:38:14.285629 7247 raft_consensus.cc:2258] T 3203988bbf1942fbacc78dc10f0fb48e P 69be14cecf364c638c3881e7f9f90cbc [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 18:38:14.285759 20232 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJW69tn/build/debug/bin/kudu with pid 6783
I20260806 18:38:14.288715 7246 raft_consensus.cc:2258] T 3203988bbf1942fbacc78dc10f0fb48e P 1d66bfcc0f164b148cc6c8be51f47a9a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 18:38:14.290145 7246 ts_tablet_manager.cc:1905] T 3203988bbf1942fbacc78dc10f0fb48e P 1d66bfcc0f164b148cc6c8be51f47a9a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 18:38:14.292919 7246 ts_tablet_manager.cc:1918] T 3203988bbf1942fbacc78dc10f0fb48e P 1d66bfcc0f164b148cc6c8be51f47a9a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 18:38:14.293023 7246 log.cc:1196] T 3203988bbf1942fbacc78dc10f0fb48e P 1d66bfcc0f164b148cc6c8be51f47a9a: Deleting WAL directory at /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786041246094603-20232-0/minicluster-data/ts-1/wal/wals/3203988bbf1942fbacc78dc10f0fb48e
I20260806 18:38:14.293288 7248 tablet_replica.cc:330] T 3203988bbf1942fbacc78dc10f0fb48e P a598fd9d7cf94ccc9f339d2560f35ab8: stopping tablet replica
I20260806 18:38:14.293344 7246 ts_tablet_manager.cc:1939] T 3203988bbf1942fbacc78dc10f0fb48e P 1d66bfcc0f164b148cc6c8be51f47a9a: Deleting consensus metadata
I20260806 18:38:14.293706 7248 raft_consensus.cc:2229] T 3203988bbf1942fbacc78dc10f0fb48e P a598fd9d7cf94ccc9f339d2560f35ab8 [term 1 FOLLOWER]: Raft consensus shutting down.
W20260806 18:38:14.293877 6706 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.19.194.1:37755 (error 108)
I20260806 18:38:14.294111 7248 raft_consensus.cc:2258] T 3203988bbf1942fbacc78dc10f0fb48e P a598fd9d7cf94ccc9f339d2560f35ab8 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 18:38:14.294412 6705 catalog_manager.cc:4841] TS 1d66bfcc0f164b148cc6c8be51f47a9a (127.19.194.2:45107): tablet 3203988bbf1942fbacc78dc10f0fb48e (table test-table [id=68982399de934429bfef89b9a8a1b479]) successfully deleted
I20260806 18:38:14.295217 7219 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:38:14.294906 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.194.1:37755 (local address 127.0.0.1:37340)
0806 18:38:14.295023 (+ 117us) negotiation.cc:107] Waiting for socket to connect
0806 18:38:14.295102 (+ 79us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.194.1:37755: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":36}
I20260806 18:38:14.295307 20232 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJW69tn/build/debug/bin/kudu with pid 6926
I20260806 18:38:14.296265 7248 ts_tablet_manager.cc:1905] T 3203988bbf1942fbacc78dc10f0fb48e P a598fd9d7cf94ccc9f339d2560f35ab8: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260806 18:38:14.297749 6706 catalog_manager.cc:4584] TS 69be14cecf364c638c3881e7f9f90cbc (127.19.194.1:37755): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 3203988bbf1942fbacc78dc10f0fb48e: Network error: Client connection negotiation failed: client connection to 127.19.194.1:37755: connect: Connection refused (error 111)
I20260806 18:38:14.298718 7248 ts_tablet_manager.cc:1918] T 3203988bbf1942fbacc78dc10f0fb48e P a598fd9d7cf94ccc9f339d2560f35ab8: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 18:38:14.298830 7248 log.cc:1196] T 3203988bbf1942fbacc78dc10f0fb48e P a598fd9d7cf94ccc9f339d2560f35ab8: Deleting WAL directory at /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786041246094603-20232-0/minicluster-data/ts-2/wal/wals/3203988bbf1942fbacc78dc10f0fb48e
I20260806 18:38:14.299185 7248 ts_tablet_manager.cc:1939] T 3203988bbf1942fbacc78dc10f0fb48e P a598fd9d7cf94ccc9f339d2560f35ab8: Deleting consensus metadata
I20260806 18:38:14.300185 6706 catalog_manager.cc:4841] TS a598fd9d7cf94ccc9f339d2560f35ab8 (127.19.194.3:37707): tablet 3203988bbf1942fbacc78dc10f0fb48e (table test-table [id=68982399de934429bfef89b9a8a1b479]) successfully deleted
I20260806 18:38:14.303241 20232 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJW69tn/build/debug/bin/kudu with pid 7069
I20260806 18:38:14.309803 20232 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJW69tn/build/debug/bin/kudu with pid 6684
2026-08-06T18:38:14Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/2 (2313 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/3
2026-08-06T18:38:14Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T18:38:14Z Disabled control of system clock
I20260806 18:38:14.340947 20232 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJW69tn/build/debug/bin/kudu
/tmp/dist-test-taskJW69tn/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786041246094603-20232-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786041246094603-20232-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786041246094603-20232-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786041246094603-20232-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.19.194.62:38599
--webserver_interface=127.19.194.62
--webserver_port=0
--builtin_ntp_servers=127.19.194.20:36801
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.19.194.62:38599
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260806 18:38:14.448695 7253 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 18:38:14.448935 7253 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 18:38:14.448969 7253 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 18:38:14.452833 7253 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 18:38:14.452898 7253 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 18:38:14.452913 7253 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 18:38:14.452929 7253 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 18:38:14.452944 7253 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 18:38:14.457499 7253 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.194.20:36801
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786041246094603-20232-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786041246094603-20232-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.19.194.62:38599
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.19.194.62:38599
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786041246094603-20232-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.19.194.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7253
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786041246094603-20232-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 17:59:13 UTC on ccycloud.quasar-wvsnzx.root.comops.site
build id 4571028
I20260806 18:38:14.458413 7253 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 18:38:14.459831 7253 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 18:38:14.468187 7259 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 18:38:14.468241 7253 server_base.cc:1034] running on GCE node
W20260806 18:38:14.468205 7258 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:38:14.468192 7261 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 18:38:14.468947 7253 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 18:38:14.469955 7253 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 18:38:14.471189 7253 hybrid_clock.cc:648] HybridClock initialized: now 1786041494471154 us; error 38 us; skew 500 ppm
I20260806 18:38:14.472929 7253 webserver.cc:460] Webserver started at http://127.19.194.62:39961/ using document root <none> and password file <none>
I20260806 18:38:14.473479 7253 fs_manager.cc:359] Metadata directory not provided
I20260806 18:38:14.473569 7253 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 18:38:14.473795 7253 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 18:38:14.475847 7253 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786041246094603-20232-0/minicluster-data/master-0/data/instance:
uuid: "77b7edaf8ec244c3aeda550fa438fa5a"
format_stamp: "Formatted at 2026-08-06 18:38:14 on dist-test-slave-1xrh"
server_key: "a8926e58d0cabce0661cb66a0d4821de"
server_key_iv: "aadbf424fdbd3b7116e03e65cf33979d"
server_key_version: "encryptionkey@0"
I20260806 18:38:14.476409 7253 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786041246094603-20232-0/minicluster-data/master-0/wal/instance:
uuid: "77b7edaf8ec244c3aeda550fa438fa5a"
format_stamp: "Formatted at 2026-08-06 18:38:14 on dist-test-slave-1xrh"
server_key: "a8926e58d0cabce0661cb66a0d4821de"
server_key_iv: "aadbf424fdbd3b7116e03e65cf33979d"
server_key_version: "encryptionkey@0"
I20260806 18:38:14.479887 7253 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20260806 18:38:14.482072 7268 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 18:38:14.483207 7253 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260806 18:38:14.483331 7253 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786041246094603-20232-0/minicluster-data/master-0/data,/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786041246094603-20232-0/minicluster-data/master-0/wal
uuid: "77b7edaf8ec244c3aeda550fa438fa5a"
format_stamp: "Formatted at 2026-08-06 18:38:14 on dist-test-slave-1xrh"
server_key: "a8926e58d0cabce0661cb66a0d4821de"
server_key_iv: "aadbf424fdbd3b7116e03e65cf33979d"
server_key_version: "encryptionkey@0"
I20260806 18:38:14.483435 7253 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786041246094603-20232-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786041246094603-20232-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786041246094603-20232-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 18:38:14.497740 7253 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 18:38:14.498344 7253 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 18:38:14.498528 7253 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 18:38:14.505570 7253 rpc_server.cc:307] RPC server started. Bound to: 127.19.194.62:38599
I20260806 18:38:14.505581 7321 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.194.62:38599 every 8 connection(s)
I20260806 18:38:14.506477 7253 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786041246094603-20232-0/minicluster-data/master-0/data/info.pb
I20260806 18:38:14.509294 7322 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 18:38:14.514335 7322 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 77b7edaf8ec244c3aeda550fa438fa5a: Bootstrap starting.
I20260806 18:38:14.516441 20232 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJW69tn/build/debug/bin/kudu as pid 7253
I20260806 18:38:14.516546 20232 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786041246094603-20232-0/minicluster-data/master-0/wal/instance
I20260806 18:38:14.516665 7322 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 77b7edaf8ec244c3aeda550fa438fa5a: Neither blocks nor log segments found. Creating new log.
I20260806 18:38:14.516862 20232 external_mini_cluster.cc:1428] Setting key 82b84472fae096ca4c369c4027620bf4
I20260806 18:38:14.517454 7322 log.cc:824] T 00000000000000000000000000000000 P 77b7edaf8ec244c3aeda550fa438fa5a: Log is configured to *not* fsync() on all Append() calls
I20260806 18:38:14.519336 7322 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 77b7edaf8ec244c3aeda550fa438fa5a: No bootstrap required, opened a new log
I20260806 18:38:14.522567 7322 raft_consensus.cc:348] T 00000000000000000000000000000000 P 77b7edaf8ec244c3aeda550fa438fa5a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "77b7edaf8ec244c3aeda550fa438fa5a" member_type: VOTER last_known_addr { host: "127.19.194.62" port: 38599 } }
I20260806 18:38:14.522787 7322 raft_consensus.cc:374] T 00000000000000000000000000000000 P 77b7edaf8ec244c3aeda550fa438fa5a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 18:38:14.522841 7322 raft_consensus.cc:729] T 00000000000000000000000000000000 P 77b7edaf8ec244c3aeda550fa438fa5a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 77b7edaf8ec244c3aeda550fa438fa5a, State: Initialized, Role: FOLLOWER
I20260806 18:38:14.523309 7322 consensus_queue.cc:260] T 00000000000000000000000000000000 P 77b7edaf8ec244c3aeda550fa438fa5a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "77b7edaf8ec244c3aeda550fa438fa5a" member_type: VOTER last_known_addr { host: "127.19.194.62" port: 38599 } }
I20260806 18:38:14.523459 7322 raft_consensus.cc:388] T 00000000000000000000000000000000 P 77b7edaf8ec244c3aeda550fa438fa5a [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 18:38:14.523533 7322 raft_consensus.cc:482] T 00000000000000000000000000000000 P 77b7edaf8ec244c3aeda550fa438fa5a [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 18:38:14.523658 7322 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 77b7edaf8ec244c3aeda550fa438fa5a [term 0 FOLLOWER]: Advancing to term 1
I20260806 18:38:14.524809 7322 raft_consensus.cc:504] T 00000000000000000000000000000000 P 77b7edaf8ec244c3aeda550fa438fa5a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "77b7edaf8ec244c3aeda550fa438fa5a" member_type: VOTER last_known_addr { host: "127.19.194.62" port: 38599 } }
I20260806 18:38:14.525154 7322 leader_election.cc:302] T 00000000000000000000000000000000 P 77b7edaf8ec244c3aeda550fa438fa5a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 77b7edaf8ec244c3aeda550fa438fa5a; no voters:
I20260806 18:38:14.525465 7322 leader_election.cc:288] T 00000000000000000000000000000000 P 77b7edaf8ec244c3aeda550fa438fa5a [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 18:38:14.525580 7328 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 77b7edaf8ec244c3aeda550fa438fa5a [term 1 FOLLOWER]: Leader election won for term 1
I20260806 18:38:14.525831 7327 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:38:14.518004 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55512 (local address 127.19.194.62:38599)
0806 18:38:14.518615 (+ 611us) server_negotiation.cc:206] Beginning negotiation
0806 18:38:14.518631 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0806 18:38:14.518664 (+ 33us) server_negotiation.cc:407] Connection header received
0806 18:38:14.519331 (+ 667us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:38:14.519348 (+ 17us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:38:14.519705 (+ 357us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:38:14.520027 (+ 322us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:38:14.520434 (+ 407us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:38:14.521351 (+ 917us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:38:14.522242 (+ 891us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:38:14.522686 (+ 444us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:38:14.522857 (+ 171us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:38:14.523734 (+ 877us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:38:14.523741 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:38:14.523757 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 18:38:14.524247 (+ 490us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:38:14.524323 (+ 76us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 18:38:14.524349 (+ 26us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:38:14.524479 (+ 130us) server_negotiation.cc:1035] Waiting for connection context
0806 18:38:14.524577 (+ 98us) server_negotiation.cc:299] Negotiation successful
0806 18:38:14.524849 (+ 272us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":271,"thread_start_us":134,"threads_started":1}
I20260806 18:38:14.525842 7328 raft_consensus.cc:686] T 00000000000000000000000000000000 P 77b7edaf8ec244c3aeda550fa438fa5a [term 1 LEADER]: Becoming Leader. State: Replica: 77b7edaf8ec244c3aeda550fa438fa5a, State: Running, Role: LEADER
I20260806 18:38:14.526180 7328 consensus_queue.cc:237] T 00000000000000000000000000000000 P 77b7edaf8ec244c3aeda550fa438fa5a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "77b7edaf8ec244c3aeda550fa438fa5a" member_type: VOTER last_known_addr { host: "127.19.194.62" port: 38599 } }
I20260806 18:38:14.526607 7322 sys_catalog.cc:564] T 00000000000000000000000000000000 P 77b7edaf8ec244c3aeda550fa438fa5a [sys.catalog]: configured and running, proceeding with master startup.
I20260806 18:38:14.527311 7330 sys_catalog.cc:455] T 00000000000000000000000000000000 P 77b7edaf8ec244c3aeda550fa438fa5a [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "77b7edaf8ec244c3aeda550fa438fa5a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "77b7edaf8ec244c3aeda550fa438fa5a" member_type: VOTER last_known_addr { host: "127.19.194.62" port: 38599 } } }
I20260806 18:38:14.527428 7330 sys_catalog.cc:458] T 00000000000000000000000000000000 P 77b7edaf8ec244c3aeda550fa438fa5a [sys.catalog]: This master's current role is: LEADER
I20260806 18:38:14.527648 7330 sys_catalog.cc:455] T 00000000000000000000000000000000 P 77b7edaf8ec244c3aeda550fa438fa5a [sys.catalog]: SysCatalogTable state changed. Reason: New leader 77b7edaf8ec244c3aeda550fa438fa5a. Latest consensus state: current_term: 1 leader_uuid: "77b7edaf8ec244c3aeda550fa438fa5a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "77b7edaf8ec244c3aeda550fa438fa5a" member_type: VOTER last_known_addr { host: "127.19.194.62" port: 38599 } } }
I20260806 18:38:14.527719 7330 sys_catalog.cc:458] T 00000000000000000000000000000000 P 77b7edaf8ec244c3aeda550fa438fa5a [sys.catalog]: This master's current role is: LEADER
I20260806 18:38:14.528004 7337 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 18:38:14.530956 7337 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 18:38:14.536124 7337 catalog_manager.cc:1347] Generated new cluster ID: 9355a4965900453fbfea579f00380bda
I20260806 18:38:14.536185 7337 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 18:38:14.549984 7337 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 18:38:14.550837 7337 catalog_manager.cc:1504] Loading token signing keys...
I20260806 18:38:14.561981 7337 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 77b7edaf8ec244c3aeda550fa438fa5a: Generated new TSK 0
I20260806 18:38:14.562608 7337 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 18:38:14.567989 20232 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJW69tn/build/debug/bin/kudu
/tmp/dist-test-taskJW69tn/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786041246094603-20232-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786041246094603-20232-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786041246094603-20232-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786041246094603-20232-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.19.194.1:0
--local_ip_for_outbound_sockets=127.19.194.1
--webserver_interface=127.19.194.1
--webserver_port=0
--tserver_master_addrs=127.19.194.62:38599
--builtin_ntp_servers=127.19.194.20:36801
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260806 18:38:14.677970 7349 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 18:38:14.678225 7349 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 18:38:14.678294 7349 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 18:38:14.682211 7349 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 18:38:14.682287 7349 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 18:38:14.682374 7349 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.19.194.1
I20260806 18:38:14.687256 7349 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.194.20:36801
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786041246094603-20232-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786041246094603-20232-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.19.194.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786041246094603-20232-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.19.194.1
--webserver_port=0
--tserver_master_addrs=127.19.194.62:38599
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7349
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.19.194.1
--log_dir=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786041246094603-20232-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 17:59:13 UTC on ccycloud.quasar-wvsnzx.root.comops.site
build id 4571028
I20260806 18:38:14.688297 7349 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 18:38:14.689610 7349 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 18:38:14.699040 7355 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 18:38:14.699087 7349 server_base.cc:1034] running on GCE node
W20260806 18:38:14.699052 7357 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:38:14.699050 7354 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 18:38:14.699712 7349 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 18:38:14.700299 7349 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 18:38:14.701470 7349 hybrid_clock.cc:648] HybridClock initialized: now 1786041494701448 us; error 52 us; skew 500 ppm
I20260806 18:38:14.703256 7349 webserver.cc:460] Webserver started at http://127.19.194.1:39121/ using document root <none> and password file <none>
I20260806 18:38:14.703758 7349 fs_manager.cc:359] Metadata directory not provided
I20260806 18:38:14.703827 7349 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 18:38:14.703994 7349 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 18:38:14.706000 7349 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786041246094603-20232-0/minicluster-data/ts-0/data/instance:
uuid: "50a5e173eeed4f9eaf2225222fae35b2"
format_stamp: "Formatted at 2026-08-06 18:38:14 on dist-test-slave-1xrh"
server_key: "08c6b3ff3e10832debb2cf3d9c72627f"
server_key_iv: "56efd9b9f422ffd97d495f64e33f6797"
server_key_version: "encryptionkey@0"
I20260806 18:38:14.706489 7349 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786041246094603-20232-0/minicluster-data/ts-0/wal/instance:
uuid: "50a5e173eeed4f9eaf2225222fae35b2"
format_stamp: "Formatted at 2026-08-06 18:38:14 on dist-test-slave-1xrh"
server_key: "08c6b3ff3e10832debb2cf3d9c72627f"
server_key_iv: "56efd9b9f422ffd97d495f64e33f6797"
server_key_version: "encryptionkey@0"
I20260806 18:38:14.710373 7349 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.002s sys 0.005s
I20260806 18:38:14.712658 7363 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 18:38:14.713802 7349 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260806 18:38:14.713944 7349 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786041246094603-20232-0/minicluster-data/ts-0/data,/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786041246094603-20232-0/minicluster-data/ts-0/wal
uuid: "50a5e173eeed4f9eaf2225222fae35b2"
format_stamp: "Formatted at 2026-08-06 18:38:14 on dist-test-slave-1xrh"
server_key: "08c6b3ff3e10832debb2cf3d9c72627f"
server_key_iv: "56efd9b9f422ffd97d495f64e33f6797"
server_key_version: "encryptionkey@0"
I20260806 18:38:14.714048 7349 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786041246094603-20232-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786041246094603-20232-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786041246094603-20232-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 18:38:14.736188 7349 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 18:38:14.736819 7349 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 18:38:14.737010 7349 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 18:38:14.737586 7349 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 18:38:14.738535 7349 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 18:38:14.738581 7349 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 18:38:14.738648 7349 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 18:38:14.738679 7349 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 18:38:14.747692 7349 rpc_server.cc:307] RPC server started. Bound to: 127.19.194.1:35909
I20260806 18:38:14.747725 7484 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.194.1:35909 every 8 connection(s)
I20260806 18:38:14.748517 7349 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786041246094603-20232-0/minicluster-data/ts-0/data/info.pb
I20260806 18:38:14.754982 20232 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJW69tn/build/debug/bin/kudu as pid 7349
I20260806 18:38:14.755164 20232 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786041246094603-20232-0/minicluster-data/ts-0/wal/instance
I20260806 18:38:14.755470 20232 external_mini_cluster.cc:1428] Setting key 22ec99d5143aa907c198e517b6584855
I20260806 18:38:14.757349 20232 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJW69tn/build/debug/bin/kudu
/tmp/dist-test-taskJW69tn/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786041246094603-20232-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786041246094603-20232-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786041246094603-20232-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786041246094603-20232-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.19.194.2:0
--local_ip_for_outbound_sockets=127.19.194.2
--webserver_interface=127.19.194.2
--webserver_port=0
--tserver_master_addrs=127.19.194.62:38599
--builtin_ntp_servers=127.19.194.20:36801
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260806 18:38:14.757618 7327 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:38:14.750424 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.194.1:43641 (local address 127.19.194.62:38599)
0806 18:38:14.750582 (+ 158us) server_negotiation.cc:206] Beginning negotiation
0806 18:38:14.750586 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 18:38:14.751619 (+ 1033us) server_negotiation.cc:407] Connection header received
0806 18:38:14.752921 (+ 1302us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:38:14.752924 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:38:14.752969 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:38:14.753040 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:38:14.753845 (+ 805us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:38:14.754372 (+ 527us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:38:14.755576 (+ 1204us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:38:14.755804 (+ 228us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:38:14.755941 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:38:14.757111 (+ 1170us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:38:14.757115 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:38:14.757116 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 18:38:14.757127 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:38:14.757146 (+ 19us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 18:38:14.757154 (+ 8us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:38:14.757228 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0806 18:38:14.757400 (+ 172us) server_negotiation.cc:299] Negotiation successful
0806 18:38:14.757462 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260806 18:38:14.758383 7487 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:38:14.750780 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.194.62:38599 (local address 127.19.194.1:43641)
0806 18:38:14.751473 (+ 693us) negotiation.cc:107] Waiting for socket to connect
0806 18:38:14.751512 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0806 18:38:14.752741 (+ 1229us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:38:14.753190 (+ 449us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:38:14.753196 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:38:14.753270 (+ 74us) client_negotiation.cc:189] Negotiated authn=SASL
0806 18:38:14.753725 (+ 455us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:38:14.753737 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:38:14.754483 (+ 746us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:38:14.754486 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:38:14.755450 (+ 964us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:38:14.755460 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:38:14.755972 (+ 512us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:38:14.755975 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:38:14.756174 (+ 199us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:38:14.756823 (+ 649us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0806 18:38:14.756825 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0806 18:38:14.756877 (+ 52us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0806 18:38:14.756960 (+ 83us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 18:38:14.757232 (+ 272us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 18:38:14.757243 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 18:38:14.757252 (+ 9us) client_negotiation.cc:769] Sending connection context
0806 18:38:14.757394 (+ 142us) client_negotiation.cc:240] Negotiation successful
0806 18:38:14.757673 (+ 279us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":259,"thread_start_us":113,"threads_started":1}
I20260806 18:38:14.759946 7485 heartbeater.cc:347] Connected to a master server at 127.19.194.62:38599
I20260806 18:38:14.760196 7485 heartbeater.cc:464] Registering TS with master...
I20260806 18:38:14.760701 7485 heartbeater.cc:511] Master 127.19.194.62:38599 requested a full tablet report, sending...
I20260806 18:38:14.762122 7285 ts_manager.cc:194] Registered new tserver with Master: 50a5e173eeed4f9eaf2225222fae35b2 (127.19.194.1:35909)
I20260806 18:38:14.763338 7285 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.19.194.1:43641
W20260806 18:38:14.867981 7488 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 18:38:14.868247 7488 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 18:38:14.868315 7488 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 18:38:14.872207 7488 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 18:38:14.872282 7488 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 18:38:14.872363 7488 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.19.194.2
I20260806 18:38:14.877094 7488 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.194.20:36801
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786041246094603-20232-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786041246094603-20232-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.19.194.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786041246094603-20232-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.19.194.2
--webserver_port=0
--tserver_master_addrs=127.19.194.62:38599
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7488
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.19.194.2
--log_dir=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786041246094603-20232-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 17:59:13 UTC on ccycloud.quasar-wvsnzx.root.comops.site
build id 4571028
I20260806 18:38:14.878142 7488 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 18:38:14.879508 7488 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 18:38:14.889037 7494 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:38:14.889030 7496 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 18:38:14.889125 7488 server_base.cc:1034] running on GCE node
W20260806 18:38:14.889032 7493 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 18:38:14.889663 7488 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 18:38:14.890254 7488 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 18:38:14.891443 7488 hybrid_clock.cc:648] HybridClock initialized: now 1786041494891433 us; error 26 us; skew 500 ppm
I20260806 18:38:14.893198 7488 webserver.cc:460] Webserver started at http://127.19.194.2:33487/ using document root <none> and password file <none>
I20260806 18:38:14.893822 7488 fs_manager.cc:359] Metadata directory not provided
I20260806 18:38:14.893913 7488 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 18:38:14.894119 7488 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 18:38:14.896152 7488 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786041246094603-20232-0/minicluster-data/ts-1/data/instance:
uuid: "80b05b9901c94886a0122923c8e393f7"
format_stamp: "Formatted at 2026-08-06 18:38:14 on dist-test-slave-1xrh"
server_key: "9783cfa8b3855099213c33f42a45d599"
server_key_iv: "af1fb65e948e42c09878bc6ea1beccc2"
server_key_version: "encryptionkey@0"
I20260806 18:38:14.896706 7488 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786041246094603-20232-0/minicluster-data/ts-1/wal/instance:
uuid: "80b05b9901c94886a0122923c8e393f7"
format_stamp: "Formatted at 2026-08-06 18:38:14 on dist-test-slave-1xrh"
server_key: "9783cfa8b3855099213c33f42a45d599"
server_key_iv: "af1fb65e948e42c09878bc6ea1beccc2"
server_key_version: "encryptionkey@0"
I20260806 18:38:14.900171 7488 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.001s sys 0.004s
I20260806 18:38:14.902489 7502 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 18:38:14.903688 7488 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260806 18:38:14.903848 7488 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786041246094603-20232-0/minicluster-data/ts-1/data,/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786041246094603-20232-0/minicluster-data/ts-1/wal
uuid: "80b05b9901c94886a0122923c8e393f7"
format_stamp: "Formatted at 2026-08-06 18:38:14 on dist-test-slave-1xrh"
server_key: "9783cfa8b3855099213c33f42a45d599"
server_key_iv: "af1fb65e948e42c09878bc6ea1beccc2"
server_key_version: "encryptionkey@0"
I20260806 18:38:14.903968 7488 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786041246094603-20232-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786041246094603-20232-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786041246094603-20232-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 18:38:14.936394 7488 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 18:38:14.937130 7488 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 18:38:14.937337 7488 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 18:38:14.937950 7488 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 18:38:14.938900 7488 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 18:38:14.938972 7488 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 18:38:14.939049 7488 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 18:38:14.939093 7488 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 18:38:14.948066 7488 rpc_server.cc:307] RPC server started. Bound to: 127.19.194.2:42143
I20260806 18:38:14.948125 7616 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.194.2:42143 every 8 connection(s)
I20260806 18:38:14.948915 7488 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786041246094603-20232-0/minicluster-data/ts-1/data/info.pb
I20260806 18:38:14.954840 20232 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJW69tn/build/debug/bin/kudu as pid 7488
I20260806 18:38:14.954964 20232 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786041246094603-20232-0/minicluster-data/ts-1/wal/instance
I20260806 18:38:14.955304 20232 external_mini_cluster.cc:1428] Setting key bda9e58299af7ab30b1619de006fffb3
I20260806 18:38:14.957370 20232 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJW69tn/build/debug/bin/kudu
/tmp/dist-test-taskJW69tn/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786041246094603-20232-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786041246094603-20232-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786041246094603-20232-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786041246094603-20232-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.19.194.3:0
--local_ip_for_outbound_sockets=127.19.194.3
--webserver_interface=127.19.194.3
--webserver_port=0
--tserver_master_addrs=127.19.194.62:38599
--builtin_ntp_servers=127.19.194.20:36801
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260806 18:38:14.958384 7327 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:38:14.950616 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.194.2:44281 (local address 127.19.194.62:38599)
0806 18:38:14.950755 (+ 139us) server_negotiation.cc:206] Beginning negotiation
0806 18:38:14.950759 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 18:38:14.951764 (+ 1005us) server_negotiation.cc:407] Connection header received
0806 18:38:14.953212 (+ 1448us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:38:14.953215 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:38:14.953293 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:38:14.953403 (+ 110us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:38:14.954265 (+ 862us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:38:14.954986 (+ 721us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:38:14.956287 (+ 1301us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:38:14.956541 (+ 254us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:38:14.956637 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:38:14.957809 (+ 1172us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:38:14.957815 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:38:14.957821 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 18:38:14.957838 (+ 17us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:38:14.957864 (+ 26us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 18:38:14.957878 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:38:14.957969 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0806 18:38:14.958102 (+ 133us) server_negotiation.cc:299] Negotiation successful
0806 18:38:14.958175 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260806 18:38:14.959164 7620 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:38:14.950981 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.194.62:38599 (local address 127.19.194.2:44281)
0806 18:38:14.951613 (+ 632us) negotiation.cc:107] Waiting for socket to connect
0806 18:38:14.951652 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0806 18:38:14.952987 (+ 1335us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:38:14.953575 (+ 588us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:38:14.953582 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:38:14.953661 (+ 79us) client_negotiation.cc:189] Negotiated authn=SASL
0806 18:38:14.954118 (+ 457us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:38:14.954129 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:38:14.955105 (+ 976us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:38:14.955108 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:38:14.956153 (+ 1045us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:38:14.956165 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:38:14.956697 (+ 532us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:38:14.956701 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:38:14.956915 (+ 214us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:38:14.957558 (+ 643us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0806 18:38:14.957560 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0806 18:38:14.957597 (+ 37us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0806 18:38:14.957657 (+ 60us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 18:38:14.957968 (+ 311us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 18:38:14.957976 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 18:38:14.957980 (+ 4us) client_negotiation.cc:769] Sending connection context
0806 18:38:14.958072 (+ 92us) client_negotiation.cc:240] Negotiation successful
0806 18:38:14.958276 (+ 204us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":262,"thread_start_us":126,"threads_started":1}
I20260806 18:38:14.960306 7617 heartbeater.cc:347] Connected to a master server at 127.19.194.62:38599
I20260806 18:38:14.960533 7617 heartbeater.cc:464] Registering TS with master...
I20260806 18:38:14.961114 7617 heartbeater.cc:511] Master 127.19.194.62:38599 requested a full tablet report, sending...
I20260806 18:38:14.962167 7285 ts_manager.cc:194] Registered new tserver with Master: 80b05b9901c94886a0122923c8e393f7 (127.19.194.2:42143)
I20260806 18:38:14.962734 7285 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.19.194.2:44281
W20260806 18:38:15.068595 7621 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 18:38:15.068850 7621 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 18:38:15.068920 7621 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 18:38:15.072813 7621 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 18:38:15.072885 7621 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 18:38:15.072973 7621 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.19.194.3
I20260806 18:38:15.077389 7621 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.194.20:36801
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786041246094603-20232-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786041246094603-20232-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.19.194.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786041246094603-20232-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.19.194.3
--webserver_port=0
--tserver_master_addrs=127.19.194.62:38599
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7621
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.19.194.3
--log_dir=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786041246094603-20232-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 17:59:13 UTC on ccycloud.quasar-wvsnzx.root.comops.site
build id 4571028
I20260806 18:38:15.078316 7621 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 18:38:15.079593 7621 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 18:38:15.088590 7627 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:38:15.088611 7628 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:38:15.088595 7630 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 18:38:15.088975 7621 server_base.cc:1034] running on GCE node
I20260806 18:38:15.089335 7621 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 18:38:15.089887 7621 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 18:38:15.091073 7621 hybrid_clock.cc:648] HybridClock initialized: now 1786041495091055 us; error 27 us; skew 500 ppm
I20260806 18:38:15.092760 7621 webserver.cc:460] Webserver started at http://127.19.194.3:41293/ using document root <none> and password file <none>
I20260806 18:38:15.093299 7621 fs_manager.cc:359] Metadata directory not provided
I20260806 18:38:15.093359 7621 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 18:38:15.093617 7621 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 18:38:15.095613 7621 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786041246094603-20232-0/minicluster-data/ts-2/data/instance:
uuid: "ff45a526a9364c59bcaa2d6ed2e6c2e2"
format_stamp: "Formatted at 2026-08-06 18:38:15 on dist-test-slave-1xrh"
server_key: "b7f3bb23912259b8ac42b7f0fdb40177"
server_key_iv: "9a32bb09ce386866ff440c152280468c"
server_key_version: "encryptionkey@0"
I20260806 18:38:15.096150 7621 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786041246094603-20232-0/minicluster-data/ts-2/wal/instance:
uuid: "ff45a526a9364c59bcaa2d6ed2e6c2e2"
format_stamp: "Formatted at 2026-08-06 18:38:15 on dist-test-slave-1xrh"
server_key: "b7f3bb23912259b8ac42b7f0fdb40177"
server_key_iv: "9a32bb09ce386866ff440c152280468c"
server_key_version: "encryptionkey@0"
I20260806 18:38:15.099534 7621 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.001s
I20260806 18:38:15.101759 7636 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 18:38:15.102921 7621 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260806 18:38:15.103062 7621 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786041246094603-20232-0/minicluster-data/ts-2/data,/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786041246094603-20232-0/minicluster-data/ts-2/wal
uuid: "ff45a526a9364c59bcaa2d6ed2e6c2e2"
format_stamp: "Formatted at 2026-08-06 18:38:15 on dist-test-slave-1xrh"
server_key: "b7f3bb23912259b8ac42b7f0fdb40177"
server_key_iv: "9a32bb09ce386866ff440c152280468c"
server_key_version: "encryptionkey@0"
I20260806 18:38:15.103205 7621 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786041246094603-20232-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786041246094603-20232-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786041246094603-20232-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 18:38:15.115620 7621 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 18:38:15.116312 7621 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 18:38:15.116511 7621 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 18:38:15.117118 7621 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 18:38:15.118105 7621 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 18:38:15.118151 7621 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 18:38:15.118220 7621 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 18:38:15.118265 7621 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 18:38:15.127408 7621 rpc_server.cc:307] RPC server started. Bound to: 127.19.194.3:46267
I20260806 18:38:15.127442 7754 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.194.3:46267 every 8 connection(s)
I20260806 18:38:15.128402 7621 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786041246094603-20232-0/minicluster-data/ts-2/data/info.pb
I20260806 18:38:15.134670 20232 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJW69tn/build/debug/bin/kudu as pid 7621
I20260806 18:38:15.134766 20232 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786041246094603-20232-0/minicluster-data/ts-2/wal/instance
I20260806 18:38:15.135152 20232 external_mini_cluster.cc:1428] Setting key 9dd99109bb08739286689ddad79e2b5d
I20260806 18:38:15.137820 7327 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:38:15.130193 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.194.3:57287 (local address 127.19.194.62:38599)
0806 18:38:15.130366 (+ 173us) server_negotiation.cc:206] Beginning negotiation
0806 18:38:15.130371 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 18:38:15.131427 (+ 1056us) server_negotiation.cc:407] Connection header received
0806 18:38:15.132832 (+ 1405us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:38:15.132834 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:38:15.132880 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:38:15.132953 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:38:15.133828 (+ 875us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:38:15.134344 (+ 516us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:38:15.135749 (+ 1405us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:38:15.136063 (+ 314us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:38:15.136187 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:38:15.137383 (+ 1196us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:38:15.137387 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:38:15.137390 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 18:38:15.137404 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:38:15.137434 (+ 30us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 18:38:15.137446 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:38:15.137522 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0806 18:38:15.137666 (+ 144us) server_negotiation.cc:299] Negotiation successful
0806 18:38:15.137713 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71}
I20260806 18:38:15.138648 7757 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:38:15.130588 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.194.62:38599 (local address 127.19.194.3:57287)
0806 18:38:15.131281 (+ 693us) negotiation.cc:107] Waiting for socket to connect
0806 18:38:15.131321 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0806 18:38:15.132642 (+ 1321us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:38:15.133111 (+ 469us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:38:15.133118 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:38:15.133198 (+ 80us) client_negotiation.cc:189] Negotiated authn=SASL
0806 18:38:15.133692 (+ 494us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:38:15.133704 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:38:15.134464 (+ 760us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:38:15.134467 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:38:15.135626 (+ 1159us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:38:15.135638 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:38:15.136235 (+ 597us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:38:15.136238 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:38:15.136419 (+ 181us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:38:15.137078 (+ 659us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0806 18:38:15.137080 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0806 18:38:15.137135 (+ 55us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0806 18:38:15.137218 (+ 83us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 18:38:15.137535 (+ 317us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 18:38:15.137543 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 18:38:15.137548 (+ 5us) client_negotiation.cc:769] Sending connection context
0806 18:38:15.137646 (+ 98us) client_negotiation.cc:240] Negotiation successful
0806 18:38:15.137851 (+ 205us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":246,"thread_start_us":114,"threads_started":1}
I20260806 18:38:15.139659 7755 heartbeater.cc:347] Connected to a master server at 127.19.194.62:38599
I20260806 18:38:15.139885 7755 heartbeater.cc:464] Registering TS with master...
I20260806 18:38:15.140359 7755 heartbeater.cc:511] Master 127.19.194.62:38599 requested a full tablet report, sending...
I20260806 18:38:15.141290 7285 ts_manager.cc:194] Registered new tserver with Master: ff45a526a9364c59bcaa2d6ed2e6c2e2 (127.19.194.3:46267)
I20260806 18:38:15.141798 7285 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.19.194.3:57287
I20260806 18:38:15.148959 20232 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260806 18:38:15.154462 7327 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:38:15.150812 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55518 (local address 127.19.194.62:38599)
0806 18:38:15.150932 (+ 120us) server_negotiation.cc:206] Beginning negotiation
0806 18:38:15.150942 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0806 18:38:15.151174 (+ 232us) server_negotiation.cc:407] Connection header received
0806 18:38:15.151296 (+ 122us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:38:15.151298 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:38:15.151333 (+ 35us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:38:15.151396 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:38:15.151656 (+ 260us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:38:15.152151 (+ 495us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:38:15.152938 (+ 787us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:38:15.153132 (+ 194us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:38:15.153203 (+ 71us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:38:15.154058 (+ 855us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:38:15.154061 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:38:15.154063 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 18:38:15.154074 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:38:15.154090 (+ 16us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 18:38:15.154097 (+ 7us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:38:15.154192 (+ 95us) server_negotiation.cc:1035] Waiting for connection context
0806 18:38:15.154333 (+ 141us) server_negotiation.cc:299] Negotiation successful
0806 18:38:15.154364 (+ 31us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260806 18:38:15.157315 7285 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:55518:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260806 18:38:15.160266 7285 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 18:38:15.173686 7770 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:38:15.168238 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.194.2:42143 (local address 127.0.0.1:58170)
0806 18:38:15.168938 (+ 700us) negotiation.cc:107] Waiting for socket to connect
0806 18:38:15.168984 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0806 18:38:15.169160 (+ 176us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:38:15.169929 (+ 769us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:38:15.169934 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:38:15.169952 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 18:38:15.170036 (+ 84us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:38:15.170039 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:38:15.171237 (+ 1198us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:38:15.171242 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:38:15.172388 (+ 1146us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:38:15.172401 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:38:15.173178 (+ 777us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:38:15.173181 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:38:15.173285 (+ 104us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:38:15.173331 (+ 46us) client_negotiation.cc:769] Sending connection context
0806 18:38:15.173433 (+ 102us) client_negotiation.cc:240] Negotiation successful
0806 18:38:15.173544 (+ 111us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":537,"thread_start_us":72,"threads_started":1}
I20260806 18:38:15.173702 7772 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:38:15.168393 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58170 (local address 127.19.194.2:42143)
0806 18:38:15.168709 (+ 316us) server_negotiation.cc:206] Beginning negotiation
0806 18:38:15.168722 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0806 18:38:15.169078 (+ 356us) server_negotiation.cc:407] Connection header received
0806 18:38:15.169369 (+ 291us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:38:15.169375 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:38:15.169566 (+ 191us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:38:15.169736 (+ 170us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 18:38:15.170152 (+ 416us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:38:15.171104 (+ 952us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:38:15.172510 (+ 1406us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:38:15.173041 (+ 531us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:38:15.173166 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:38:15.173344 (+ 178us) server_negotiation.cc:1035] Waiting for connection context
0806 18:38:15.173459 (+ 115us) server_negotiation.cc:299] Negotiation successful
0806 18:38:15.173543 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":214,"thread_start_us":125,"threads_started":1}
I20260806 18:38:15.174338 7771 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:38:15.168368 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.194.3:46267 (local address 127.0.0.1:46680)
0806 18:38:15.169024 (+ 656us) negotiation.cc:107] Waiting for socket to connect
0806 18:38:15.169038 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0806 18:38:15.169153 (+ 115us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:38:15.170598 (+ 1445us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:38:15.170601 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:38:15.170616 (+ 15us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 18:38:15.170683 (+ 67us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:38:15.170687 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:38:15.171965 (+ 1278us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:38:15.171968 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:38:15.172896 (+ 928us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:38:15.172906 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:38:15.173911 (+ 1005us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:38:15.173915 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:38:15.174025 (+ 110us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:38:15.174046 (+ 21us) client_negotiation.cc:769] Sending connection context
0806 18:38:15.174090 (+ 44us) client_negotiation.cc:240] Negotiation successful
0806 18:38:15.174152 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":584,"mutex_wait_us":13,"thread_start_us":81,"threads_started":1}
I20260806 18:38:15.175081 7774 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:38:15.169394 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46680 (local address 127.19.194.3:46267)
0806 18:38:15.170147 (+ 753us) server_negotiation.cc:206] Beginning negotiation
0806 18:38:15.170158 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0806 18:38:15.170179 (+ 21us) server_negotiation.cc:407] Connection header received
0806 18:38:15.170251 (+ 72us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:38:15.170258 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:38:15.170414 (+ 156us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:38:15.170634 (+ 220us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 18:38:15.170759 (+ 125us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:38:15.171837 (+ 1078us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:38:15.173030 (+ 1193us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:38:15.173763 (+ 733us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:38:15.174603 (+ 840us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:38:15.174775 (+ 172us) server_negotiation.cc:1035] Waiting for connection context
0806 18:38:15.174823 (+ 48us) server_negotiation.cc:299] Negotiation successful
0806 18:38:15.174905 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":659,"thread_start_us":114,"threads_started":1}
I20260806 18:38:15.175688 7773 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:38:15.168922 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.194.1:35909 (local address 127.0.0.1:43370)
0806 18:38:15.169770 (+ 848us) negotiation.cc:107] Waiting for socket to connect
0806 18:38:15.169811 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0806 18:38:15.169895 (+ 84us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:38:15.170334 (+ 439us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:38:15.170337 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:38:15.170356 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 18:38:15.170434 (+ 78us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:38:15.170439 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:38:15.171885 (+ 1446us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:38:15.171891 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:38:15.174136 (+ 2245us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:38:15.174144 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:38:15.175295 (+ 1151us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:38:15.175299 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:38:15.175392 (+ 93us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:38:15.175413 (+ 21us) client_negotiation.cc:769] Sending connection context
0806 18:38:15.175486 (+ 73us) client_negotiation.cc:240] Negotiation successful
0806 18:38:15.175558 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":785,"mutex_wait_us":42,"thread_start_us":74,"threads_started":1}
I20260806 18:38:15.176131 7775 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:38:15.169456 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43370 (local address 127.19.194.1:35909)
0806 18:38:15.169885 (+ 429us) server_negotiation.cc:206] Beginning negotiation
0806 18:38:15.169896 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0806 18:38:15.169919 (+ 23us) server_negotiation.cc:407] Connection header received
0806 18:38:15.170016 (+ 97us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:38:15.170021 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:38:15.170191 (+ 170us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:38:15.170333 (+ 142us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 18:38:15.170562 (+ 229us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:38:15.171747 (+ 1185us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:38:15.174250 (+ 2503us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:38:15.174920 (+ 670us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:38:15.175042 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:38:15.175241 (+ 199us) server_negotiation.cc:1035] Waiting for connection context
0806 18:38:15.175857 (+ 616us) server_negotiation.cc:299] Negotiation successful
0806 18:38:15.175978 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":302,"thread_start_us":109,"threads_started":1}
I20260806 18:38:15.176934 7551 tablet_service.cc:1467] Processing CreateTablet for tablet 458ddeb03772404a8ce7ce0084990d04 (DEFAULT_TABLE table=test-table [id=7472995e6d5c455c90d88fb1e116f387]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 18:38:15.177538 7687 tablet_service.cc:1467] Processing CreateTablet for tablet 458ddeb03772404a8ce7ce0084990d04 (DEFAULT_TABLE table=test-table [id=7472995e6d5c455c90d88fb1e116f387]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 18:38:15.178035 7551 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 458ddeb03772404a8ce7ce0084990d04. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 18:38:15.178355 7687 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 458ddeb03772404a8ce7ce0084990d04. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 18:38:15.178788 7415 tablet_service.cc:1467] Processing CreateTablet for tablet 458ddeb03772404a8ce7ce0084990d04 (DEFAULT_TABLE table=test-table [id=7472995e6d5c455c90d88fb1e116f387]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 18:38:15.179687 7415 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 458ddeb03772404a8ce7ce0084990d04. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 18:38:15.184096 7776 tablet_bootstrap.cc:492] T 458ddeb03772404a8ce7ce0084990d04 P ff45a526a9364c59bcaa2d6ed2e6c2e2: Bootstrap starting.
I20260806 18:38:15.185449 7777 tablet_bootstrap.cc:492] T 458ddeb03772404a8ce7ce0084990d04 P 80b05b9901c94886a0122923c8e393f7: Bootstrap starting.
I20260806 18:38:15.186195 7778 tablet_bootstrap.cc:492] T 458ddeb03772404a8ce7ce0084990d04 P 50a5e173eeed4f9eaf2225222fae35b2: Bootstrap starting.
I20260806 18:38:15.186342 7776 tablet_bootstrap.cc:654] T 458ddeb03772404a8ce7ce0084990d04 P ff45a526a9364c59bcaa2d6ed2e6c2e2: Neither blocks nor log segments found. Creating new log.
I20260806 18:38:15.187256 7776 log.cc:824] T 458ddeb03772404a8ce7ce0084990d04 P ff45a526a9364c59bcaa2d6ed2e6c2e2: Log is configured to *not* fsync() on all Append() calls
I20260806 18:38:15.187549 7777 tablet_bootstrap.cc:654] T 458ddeb03772404a8ce7ce0084990d04 P 80b05b9901c94886a0122923c8e393f7: Neither blocks nor log segments found. Creating new log.
I20260806 18:38:15.188336 7777 log.cc:824] T 458ddeb03772404a8ce7ce0084990d04 P 80b05b9901c94886a0122923c8e393f7: Log is configured to *not* fsync() on all Append() calls
I20260806 18:38:15.188731 7778 tablet_bootstrap.cc:654] T 458ddeb03772404a8ce7ce0084990d04 P 50a5e173eeed4f9eaf2225222fae35b2: Neither blocks nor log segments found. Creating new log.
I20260806 18:38:15.189399 7776 tablet_bootstrap.cc:492] T 458ddeb03772404a8ce7ce0084990d04 P ff45a526a9364c59bcaa2d6ed2e6c2e2: No bootstrap required, opened a new log
I20260806 18:38:15.189431 7778 log.cc:824] T 458ddeb03772404a8ce7ce0084990d04 P 50a5e173eeed4f9eaf2225222fae35b2: Log is configured to *not* fsync() on all Append() calls
I20260806 18:38:15.189579 7776 ts_tablet_manager.cc:1397] T 458ddeb03772404a8ce7ce0084990d04 P ff45a526a9364c59bcaa2d6ed2e6c2e2: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260806 18:38:15.190274 7777 tablet_bootstrap.cc:492] T 458ddeb03772404a8ce7ce0084990d04 P 80b05b9901c94886a0122923c8e393f7: No bootstrap required, opened a new log
I20260806 18:38:15.190457 7777 ts_tablet_manager.cc:1397] T 458ddeb03772404a8ce7ce0084990d04 P 80b05b9901c94886a0122923c8e393f7: Time spent bootstrapping tablet: real 0.005s user 0.003s sys 0.000s
I20260806 18:38:15.191159 7778 tablet_bootstrap.cc:492] T 458ddeb03772404a8ce7ce0084990d04 P 50a5e173eeed4f9eaf2225222fae35b2: No bootstrap required, opened a new log
I20260806 18:38:15.191345 7778 ts_tablet_manager.cc:1397] T 458ddeb03772404a8ce7ce0084990d04 P 50a5e173eeed4f9eaf2225222fae35b2: Time spent bootstrapping tablet: real 0.005s user 0.004s sys 0.000s
I20260806 18:38:15.193840 7776 raft_consensus.cc:348] T 458ddeb03772404a8ce7ce0084990d04 P ff45a526a9364c59bcaa2d6ed2e6c2e2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80b05b9901c94886a0122923c8e393f7" member_type: VOTER last_known_addr { host: "127.19.194.2" port: 42143 } } peers { permanent_uuid: "ff45a526a9364c59bcaa2d6ed2e6c2e2" member_type: VOTER last_known_addr { host: "127.19.194.3" port: 46267 } } peers { permanent_uuid: "50a5e173eeed4f9eaf2225222fae35b2" member_type: VOTER last_known_addr { host: "127.19.194.1" port: 35909 } }
I20260806 18:38:15.194103 7776 raft_consensus.cc:374] T 458ddeb03772404a8ce7ce0084990d04 P ff45a526a9364c59bcaa2d6ed2e6c2e2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 18:38:15.194154 7776 raft_consensus.cc:729] T 458ddeb03772404a8ce7ce0084990d04 P ff45a526a9364c59bcaa2d6ed2e6c2e2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ff45a526a9364c59bcaa2d6ed2e6c2e2, State: Initialized, Role: FOLLOWER
I20260806 18:38:15.194478 7777 raft_consensus.cc:348] T 458ddeb03772404a8ce7ce0084990d04 P 80b05b9901c94886a0122923c8e393f7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80b05b9901c94886a0122923c8e393f7" member_type: VOTER last_known_addr { host: "127.19.194.2" port: 42143 } } peers { permanent_uuid: "ff45a526a9364c59bcaa2d6ed2e6c2e2" member_type: VOTER last_known_addr { host: "127.19.194.3" port: 46267 } } peers { permanent_uuid: "50a5e173eeed4f9eaf2225222fae35b2" member_type: VOTER last_known_addr { host: "127.19.194.1" port: 35909 } }
I20260806 18:38:15.194568 7776 consensus_queue.cc:260] T 458ddeb03772404a8ce7ce0084990d04 P ff45a526a9364c59bcaa2d6ed2e6c2e2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80b05b9901c94886a0122923c8e393f7" member_type: VOTER last_known_addr { host: "127.19.194.2" port: 42143 } } peers { permanent_uuid: "ff45a526a9364c59bcaa2d6ed2e6c2e2" member_type: VOTER last_known_addr { host: "127.19.194.3" port: 46267 } } peers { permanent_uuid: "50a5e173eeed4f9eaf2225222fae35b2" member_type: VOTER last_known_addr { host: "127.19.194.1" port: 35909 } }
I20260806 18:38:15.194696 7777 raft_consensus.cc:374] T 458ddeb03772404a8ce7ce0084990d04 P 80b05b9901c94886a0122923c8e393f7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 18:38:15.194747 7777 raft_consensus.cc:729] T 458ddeb03772404a8ce7ce0084990d04 P 80b05b9901c94886a0122923c8e393f7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 80b05b9901c94886a0122923c8e393f7, State: Initialized, Role: FOLLOWER
I20260806 18:38:15.194756 7778 raft_consensus.cc:348] T 458ddeb03772404a8ce7ce0084990d04 P 50a5e173eeed4f9eaf2225222fae35b2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80b05b9901c94886a0122923c8e393f7" member_type: VOTER last_known_addr { host: "127.19.194.2" port: 42143 } } peers { permanent_uuid: "ff45a526a9364c59bcaa2d6ed2e6c2e2" member_type: VOTER last_known_addr { host: "127.19.194.3" port: 46267 } } peers { permanent_uuid: "50a5e173eeed4f9eaf2225222fae35b2" member_type: VOTER last_known_addr { host: "127.19.194.1" port: 35909 } }
I20260806 18:38:15.195006 7778 raft_consensus.cc:374] T 458ddeb03772404a8ce7ce0084990d04 P 50a5e173eeed4f9eaf2225222fae35b2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 18:38:15.195063 7778 raft_consensus.cc:729] T 458ddeb03772404a8ce7ce0084990d04 P 50a5e173eeed4f9eaf2225222fae35b2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 50a5e173eeed4f9eaf2225222fae35b2, State: Initialized, Role: FOLLOWER
I20260806 18:38:15.195346 7755 heartbeater.cc:503] Master 127.19.194.62:38599 was elected leader, sending a full tablet report...
I20260806 18:38:15.195533 7776 ts_tablet_manager.cc:1428] T 458ddeb03772404a8ce7ce0084990d04 P ff45a526a9364c59bcaa2d6ed2e6c2e2: Time spent starting tablet: real 0.006s user 0.007s sys 0.000s
I20260806 18:38:15.195518 7778 consensus_queue.cc:260] T 458ddeb03772404a8ce7ce0084990d04 P 50a5e173eeed4f9eaf2225222fae35b2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80b05b9901c94886a0122923c8e393f7" member_type: VOTER last_known_addr { host: "127.19.194.2" port: 42143 } } peers { permanent_uuid: "ff45a526a9364c59bcaa2d6ed2e6c2e2" member_type: VOTER last_known_addr { host: "127.19.194.3" port: 46267 } } peers { permanent_uuid: "50a5e173eeed4f9eaf2225222fae35b2" member_type: VOTER last_known_addr { host: "127.19.194.1" port: 35909 } }
I20260806 18:38:15.196089 7485 heartbeater.cc:503] Master 127.19.194.62:38599 was elected leader, sending a full tablet report...
I20260806 18:38:15.195217 7777 consensus_queue.cc:260] T 458ddeb03772404a8ce7ce0084990d04 P 80b05b9901c94886a0122923c8e393f7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80b05b9901c94886a0122923c8e393f7" member_type: VOTER last_known_addr { host: "127.19.194.2" port: 42143 } } peers { permanent_uuid: "ff45a526a9364c59bcaa2d6ed2e6c2e2" member_type: VOTER last_known_addr { host: "127.19.194.3" port: 46267 } } peers { permanent_uuid: "50a5e173eeed4f9eaf2225222fae35b2" member_type: VOTER last_known_addr { host: "127.19.194.1" port: 35909 } }
I20260806 18:38:15.196373 7778 ts_tablet_manager.cc:1428] T 458ddeb03772404a8ce7ce0084990d04 P 50a5e173eeed4f9eaf2225222fae35b2: Time spent starting tablet: real 0.005s user 0.004s sys 0.000s
I20260806 18:38:15.196841 7617 heartbeater.cc:503] Master 127.19.194.62:38599 was elected leader, sending a full tablet report...
I20260806 18:38:15.197978 7777 ts_tablet_manager.cc:1428] T 458ddeb03772404a8ce7ce0084990d04 P 80b05b9901c94886a0122923c8e393f7: Time spent starting tablet: real 0.007s user 0.007s sys 0.000s
W20260806 18:38:15.200093 7619 tablet.cc:2369] T 458ddeb03772404a8ce7ce0084990d04 P 80b05b9901c94886a0122923c8e393f7: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260806 18:38:15.249672 7486 tablet.cc:2369] T 458ddeb03772404a8ce7ce0084990d04 P 50a5e173eeed4f9eaf2225222fae35b2: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260806 18:38:15.379460 7756 tablet.cc:2369] T 458ddeb03772404a8ce7ce0084990d04 P ff45a526a9364c59bcaa2d6ed2e6c2e2: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 18:38:15.384212 7784 raft_consensus.cc:482] T 458ddeb03772404a8ce7ce0084990d04 P 50a5e173eeed4f9eaf2225222fae35b2 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 18:38:15.384394 7784 raft_consensus.cc:504] T 458ddeb03772404a8ce7ce0084990d04 P 50a5e173eeed4f9eaf2225222fae35b2 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80b05b9901c94886a0122923c8e393f7" member_type: VOTER last_known_addr { host: "127.19.194.2" port: 42143 } } peers { permanent_uuid: "ff45a526a9364c59bcaa2d6ed2e6c2e2" member_type: VOTER last_known_addr { host: "127.19.194.3" port: 46267 } } peers { permanent_uuid: "50a5e173eeed4f9eaf2225222fae35b2" member_type: VOTER last_known_addr { host: "127.19.194.1" port: 35909 } }
I20260806 18:38:15.385555 7784 leader_election.cc:288] T 458ddeb03772404a8ce7ce0084990d04 P 50a5e173eeed4f9eaf2225222fae35b2 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 80b05b9901c94886a0122923c8e393f7 (127.19.194.2:42143), ff45a526a9364c59bcaa2d6ed2e6c2e2 (127.19.194.3:46267)
I20260806 18:38:15.389782 7772 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:38:15.385890 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.194.1:46183 (local address 127.19.194.2:42143)
0806 18:38:15.386029 (+ 139us) server_negotiation.cc:206] Beginning negotiation
0806 18:38:15.386034 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 18:38:15.386252 (+ 218us) server_negotiation.cc:407] Connection header received
0806 18:38:15.386373 (+ 121us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:38:15.386377 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:38:15.386461 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:38:15.386546 (+ 85us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 18:38:15.386848 (+ 302us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:38:15.387569 (+ 721us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:38:15.388787 (+ 1218us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:38:15.389183 (+ 396us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:38:15.389300 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:38:15.389367 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0806 18:38:15.389587 (+ 220us) server_negotiation.cc:299] Negotiation successful
0806 18:38:15.389652 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260806 18:38:15.389789 7774 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:38:15.385962 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.194.1:38117 (local address 127.19.194.3:46267)
0806 18:38:15.386093 (+ 131us) server_negotiation.cc:206] Beginning negotiation
0806 18:38:15.386100 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 18:38:15.386278 (+ 178us) server_negotiation.cc:407] Connection header received
0806 18:38:15.386405 (+ 127us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:38:15.386408 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:38:15.386494 (+ 86us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:38:15.386566 (+ 72us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 18:38:15.386976 (+ 410us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:38:15.387575 (+ 599us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:38:15.388765 (+ 1190us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:38:15.389188 (+ 423us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:38:15.389330 (+ 142us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:38:15.389378 (+ 48us) server_negotiation.cc:1035] Waiting for connection context
0806 18:38:15.389617 (+ 239us) server_negotiation.cc:299] Negotiation successful
0806 18:38:15.389691 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260806 18:38:15.389796 7787 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:38:15.385873 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.194.3:46267 (local address 127.19.194.1:38117)
0806 18:38:15.386199 (+ 326us) negotiation.cc:107] Waiting for socket to connect
0806 18:38:15.386212 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0806 18:38:15.386288 (+ 76us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:38:15.386677 (+ 389us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:38:15.386682 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:38:15.386706 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 18:38:15.386840 (+ 134us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:38:15.386847 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:38:15.387711 (+ 864us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:38:15.387714 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:38:15.388637 (+ 923us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:38:15.388646 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:38:15.389336 (+ 690us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:38:15.389339 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:38:15.389472 (+ 133us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:38:15.389496 (+ 24us) client_negotiation.cc:769] Sending connection context
0806 18:38:15.389584 (+ 88us) client_negotiation.cc:240] Negotiation successful
0806 18:38:15.389657 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":263,"mutex_wait_us":55,"thread_start_us":78,"threads_started":1}
I20260806 18:38:15.389806 7786 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:38:15.385756 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.194.2:42143 (local address 127.19.194.1:46183)
0806 18:38:15.386111 (+ 355us) negotiation.cc:107] Waiting for socket to connect
0806 18:38:15.386126 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0806 18:38:15.386251 (+ 125us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:38:15.386592 (+ 341us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:38:15.386594 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:38:15.386612 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 18:38:15.386724 (+ 112us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:38:15.386728 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:38:15.387692 (+ 964us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:38:15.387697 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:38:15.388645 (+ 948us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:38:15.388655 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:38:15.389305 (+ 650us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:38:15.389309 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:38:15.389464 (+ 155us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:38:15.389489 (+ 25us) client_negotiation.cc:769] Sending connection context
0806 18:38:15.389549 (+ 60us) client_negotiation.cc:240] Negotiation successful
0806 18:38:15.389641 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":285,"mutex_wait_us":110,"thread_start_us":98,"threads_started":1}
I20260806 18:38:15.390496 7571 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "458ddeb03772404a8ce7ce0084990d04" candidate_uuid: "50a5e173eeed4f9eaf2225222fae35b2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "80b05b9901c94886a0122923c8e393f7" is_pre_election: true
I20260806 18:38:15.390517 7708 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "458ddeb03772404a8ce7ce0084990d04" candidate_uuid: "50a5e173eeed4f9eaf2225222fae35b2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ff45a526a9364c59bcaa2d6ed2e6c2e2" is_pre_election: true
I20260806 18:38:15.390779 7571 raft_consensus.cc:2454] T 458ddeb03772404a8ce7ce0084990d04 P 80b05b9901c94886a0122923c8e393f7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 50a5e173eeed4f9eaf2225222fae35b2 in term 0.
I20260806 18:38:15.390785 7708 raft_consensus.cc:2454] T 458ddeb03772404a8ce7ce0084990d04 P ff45a526a9364c59bcaa2d6ed2e6c2e2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 50a5e173eeed4f9eaf2225222fae35b2 in term 0.
I20260806 18:38:15.391279 7365 leader_election.cc:302] T 458ddeb03772404a8ce7ce0084990d04 P 50a5e173eeed4f9eaf2225222fae35b2 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 50a5e173eeed4f9eaf2225222fae35b2, ff45a526a9364c59bcaa2d6ed2e6c2e2; no voters:
I20260806 18:38:15.391571 7784 raft_consensus.cc:2781] T 458ddeb03772404a8ce7ce0084990d04 P 50a5e173eeed4f9eaf2225222fae35b2 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 18:38:15.391654 7784 raft_consensus.cc:482] T 458ddeb03772404a8ce7ce0084990d04 P 50a5e173eeed4f9eaf2225222fae35b2 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 18:38:15.391686 7784 raft_consensus.cc:3037] T 458ddeb03772404a8ce7ce0084990d04 P 50a5e173eeed4f9eaf2225222fae35b2 [term 0 FOLLOWER]: Advancing to term 1
I20260806 18:38:15.392740 7784 raft_consensus.cc:504] T 458ddeb03772404a8ce7ce0084990d04 P 50a5e173eeed4f9eaf2225222fae35b2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80b05b9901c94886a0122923c8e393f7" member_type: VOTER last_known_addr { host: "127.19.194.2" port: 42143 } } peers { permanent_uuid: "ff45a526a9364c59bcaa2d6ed2e6c2e2" member_type: VOTER last_known_addr { host: "127.19.194.3" port: 46267 } } peers { permanent_uuid: "50a5e173eeed4f9eaf2225222fae35b2" member_type: VOTER last_known_addr { host: "127.19.194.1" port: 35909 } }
I20260806 18:38:15.393159 7784 leader_election.cc:288] T 458ddeb03772404a8ce7ce0084990d04 P 50a5e173eeed4f9eaf2225222fae35b2 [CANDIDATE]: Term 1 election: Requested vote from peers 80b05b9901c94886a0122923c8e393f7 (127.19.194.2:42143), ff45a526a9364c59bcaa2d6ed2e6c2e2 (127.19.194.3:46267)
I20260806 18:38:15.393482 7571 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "458ddeb03772404a8ce7ce0084990d04" candidate_uuid: "50a5e173eeed4f9eaf2225222fae35b2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "80b05b9901c94886a0122923c8e393f7"
I20260806 18:38:15.393529 7708 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "458ddeb03772404a8ce7ce0084990d04" candidate_uuid: "50a5e173eeed4f9eaf2225222fae35b2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ff45a526a9364c59bcaa2d6ed2e6c2e2"
I20260806 18:38:15.393591 7571 raft_consensus.cc:3037] T 458ddeb03772404a8ce7ce0084990d04 P 80b05b9901c94886a0122923c8e393f7 [term 0 FOLLOWER]: Advancing to term 1
I20260806 18:38:15.393659 7708 raft_consensus.cc:3037] T 458ddeb03772404a8ce7ce0084990d04 P ff45a526a9364c59bcaa2d6ed2e6c2e2 [term 0 FOLLOWER]: Advancing to term 1
I20260806 18:38:15.394637 7571 raft_consensus.cc:2454] T 458ddeb03772404a8ce7ce0084990d04 P 80b05b9901c94886a0122923c8e393f7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 50a5e173eeed4f9eaf2225222fae35b2 in term 1.
I20260806 18:38:15.394744 7708 raft_consensus.cc:2454] T 458ddeb03772404a8ce7ce0084990d04 P ff45a526a9364c59bcaa2d6ed2e6c2e2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 50a5e173eeed4f9eaf2225222fae35b2 in term 1.
I20260806 18:38:15.394994 7367 leader_election.cc:302] T 458ddeb03772404a8ce7ce0084990d04 P 50a5e173eeed4f9eaf2225222fae35b2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 50a5e173eeed4f9eaf2225222fae35b2, 80b05b9901c94886a0122923c8e393f7; no voters:
I20260806 18:38:15.395236 7784 raft_consensus.cc:2781] T 458ddeb03772404a8ce7ce0084990d04 P 50a5e173eeed4f9eaf2225222fae35b2 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 18:38:15.395431 7784 raft_consensus.cc:686] T 458ddeb03772404a8ce7ce0084990d04 P 50a5e173eeed4f9eaf2225222fae35b2 [term 1 LEADER]: Becoming Leader. State: Replica: 50a5e173eeed4f9eaf2225222fae35b2, State: Running, Role: LEADER
I20260806 18:38:15.395730 7784 consensus_queue.cc:237] T 458ddeb03772404a8ce7ce0084990d04 P 50a5e173eeed4f9eaf2225222fae35b2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80b05b9901c94886a0122923c8e393f7" member_type: VOTER last_known_addr { host: "127.19.194.2" port: 42143 } } peers { permanent_uuid: "ff45a526a9364c59bcaa2d6ed2e6c2e2" member_type: VOTER last_known_addr { host: "127.19.194.3" port: 46267 } } peers { permanent_uuid: "50a5e173eeed4f9eaf2225222fae35b2" member_type: VOTER last_known_addr { host: "127.19.194.1" port: 35909 } }
I20260806 18:38:15.398854 7283 catalog_manager.cc:5495] T 458ddeb03772404a8ce7ce0084990d04 P 50a5e173eeed4f9eaf2225222fae35b2 reported cstate change: term changed from 0 to 1, leader changed from <none> to 50a5e173eeed4f9eaf2225222fae35b2 (127.19.194.1). New cstate: current_term: 1 leader_uuid: "50a5e173eeed4f9eaf2225222fae35b2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80b05b9901c94886a0122923c8e393f7" member_type: VOTER last_known_addr { host: "127.19.194.2" port: 42143 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ff45a526a9364c59bcaa2d6ed2e6c2e2" member_type: VOTER last_known_addr { host: "127.19.194.3" port: 46267 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "50a5e173eeed4f9eaf2225222fae35b2" member_type: VOTER last_known_addr { host: "127.19.194.1" port: 35909 } health_report { overall_health: HEALTHY } } }
I20260806 18:38:15.410768 7775 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:38:15.407433 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43374 (local address 127.19.194.1:35909)
0806 18:38:15.407542 (+ 109us) server_negotiation.cc:206] Beginning negotiation
0806 18:38:15.407545 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 18:38:15.407643 (+ 98us) server_negotiation.cc:407] Connection header received
0806 18:38:15.407793 (+ 150us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:38:15.407796 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:38:15.407843 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:38:15.407944 (+ 101us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 18:38:15.408180 (+ 236us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:38:15.408720 (+ 540us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:38:15.409517 (+ 797us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:38:15.409706 (+ 189us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:38:15.409776 (+ 70us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:38:15.410010 (+ 234us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 18:38:15.410378 (+ 368us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 18:38:15.410494 (+ 116us) server_negotiation.cc:1035] Waiting for connection context
0806 18:38:15.410572 (+ 78us) server_negotiation.cc:299] Negotiation successful
0806 18:38:15.410600 (+ 28us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260806 18:38:15.415941 7708 raft_consensus.cc:1261] T 458ddeb03772404a8ce7ce0084990d04 P ff45a526a9364c59bcaa2d6ed2e6c2e2 [term 1 FOLLOWER]: Refusing update from remote peer 50a5e173eeed4f9eaf2225222fae35b2: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 18:38:15.415942 7571 raft_consensus.cc:1261] T 458ddeb03772404a8ce7ce0084990d04 P 80b05b9901c94886a0122923c8e393f7 [term 1 FOLLOWER]: Refusing update from remote peer 50a5e173eeed4f9eaf2225222fae35b2: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 18:38:15.416585 7788 consensus_queue.cc:1035] T 458ddeb03772404a8ce7ce0084990d04 P 50a5e173eeed4f9eaf2225222fae35b2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ff45a526a9364c59bcaa2d6ed2e6c2e2" member_type: VOTER last_known_addr { host: "127.19.194.3" port: 46267 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 18:38:15.416772 7784 consensus_queue.cc:1035] T 458ddeb03772404a8ce7ce0084990d04 P 50a5e173eeed4f9eaf2225222fae35b2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "80b05b9901c94886a0122923c8e393f7" member_type: VOTER last_known_addr { host: "127.19.194.2" port: 42143 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 18:38:15.424458 7793 mvcc.cc:205] Tried to move back new op lower bound from 7315625965217476608 to 7315625965143326720. Current Snapshot: MvccSnapshot[applied={T|T < 7315625965217476608}]
I20260806 18:38:15.424988 7792 mvcc.cc:205] Tried to move back new op lower bound from 7315625965217476608 to 7315625965143326720. Current Snapshot: MvccSnapshot[applied={T|T < 7315625965217476608}]
I20260806 18:38:15.432515 7283 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:55518:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 18:38:15.432642 7283 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:55518:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 18:38:15.434939 7283 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 77b7edaf8ec244c3aeda550fa438fa5a: Sending DeleteTablet for 3 replicas of tablet 458ddeb03772404a8ce7ce0084990d04
I20260806 18:38:15.435657 7551 tablet_service.cc:1514] Processing DeleteTablet for tablet 458ddeb03772404a8ce7ce0084990d04 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 18:38:15 UTC) from {username='slave'} at 127.0.0.1:58170
I20260806 18:38:15.435706 7687 tablet_service.cc:1514] Processing DeleteTablet for tablet 458ddeb03772404a8ce7ce0084990d04 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 18:38:15 UTC) from {username='slave'} at 127.0.0.1:46680
I20260806 18:38:15.435946 7415 tablet_service.cc:1514] Processing DeleteTablet for tablet 458ddeb03772404a8ce7ce0084990d04 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 18:38:15 UTC) from {username='slave'} at 127.0.0.1:43370
I20260806 18:38:15.436340 7804 tablet_replica.cc:330] T 458ddeb03772404a8ce7ce0084990d04 P 50a5e173eeed4f9eaf2225222fae35b2: stopping tablet replica
I20260806 18:38:15.436656 20232 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJW69tn/build/debug/bin/kudu with pid 7349
I20260806 18:38:15.436647 7804 raft_consensus.cc:2229] T 458ddeb03772404a8ce7ce0084990d04 P 50a5e173eeed4f9eaf2225222fae35b2 [term 1 LEADER]: Raft consensus shutting down.
I20260806 18:38:15.439432 7803 tablet_replica.cc:330] T 458ddeb03772404a8ce7ce0084990d04 P ff45a526a9364c59bcaa2d6ed2e6c2e2: stopping tablet replica
I20260806 18:38:15.439702 7803 raft_consensus.cc:2229] T 458ddeb03772404a8ce7ce0084990d04 P ff45a526a9364c59bcaa2d6ed2e6c2e2 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 18:38:15.439970 7803 raft_consensus.cc:2258] T 458ddeb03772404a8ce7ce0084990d04 P ff45a526a9364c59bcaa2d6ed2e6c2e2 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 18:38:15.440312 7802 tablet_replica.cc:330] T 458ddeb03772404a8ce7ce0084990d04 P 80b05b9901c94886a0122923c8e393f7: stopping tablet replica
I20260806 18:38:15.440590 7802 raft_consensus.cc:2229] T 458ddeb03772404a8ce7ce0084990d04 P 80b05b9901c94886a0122923c8e393f7 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 18:38:15.440857 7802 raft_consensus.cc:2258] T 458ddeb03772404a8ce7ce0084990d04 P 80b05b9901c94886a0122923c8e393f7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 18:38:15.441725 7803 ts_tablet_manager.cc:1905] T 458ddeb03772404a8ce7ce0084990d04 P ff45a526a9364c59bcaa2d6ed2e6c2e2: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 18:38:15.442111 7802 ts_tablet_manager.cc:1905] T 458ddeb03772404a8ce7ce0084990d04 P 80b05b9901c94886a0122923c8e393f7: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 18:38:15.444630 7803 ts_tablet_manager.cc:1918] T 458ddeb03772404a8ce7ce0084990d04 P ff45a526a9364c59bcaa2d6ed2e6c2e2: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 18:38:15.444742 7803 log.cc:1196] T 458ddeb03772404a8ce7ce0084990d04 P ff45a526a9364c59bcaa2d6ed2e6c2e2: Deleting WAL directory at /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786041246094603-20232-0/minicluster-data/ts-2/wal/wals/458ddeb03772404a8ce7ce0084990d04
I20260806 18:38:15.444779 7802 ts_tablet_manager.cc:1918] T 458ddeb03772404a8ce7ce0084990d04 P 80b05b9901c94886a0122923c8e393f7: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 18:38:15.444876 7802 log.cc:1196] T 458ddeb03772404a8ce7ce0084990d04 P 80b05b9901c94886a0122923c8e393f7: Deleting WAL directory at /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786041246094603-20232-0/minicluster-data/ts-1/wal/wals/458ddeb03772404a8ce7ce0084990d04
I20260806 18:38:15.445124 7803 ts_tablet_manager.cc:1939] T 458ddeb03772404a8ce7ce0084990d04 P ff45a526a9364c59bcaa2d6ed2e6c2e2: Deleting consensus metadata
I20260806 18:38:15.445184 7802 ts_tablet_manager.cc:1939] T 458ddeb03772404a8ce7ce0084990d04 P 80b05b9901c94886a0122923c8e393f7: Deleting consensus metadata
I20260806 18:38:15.446323 7270 catalog_manager.cc:4841] TS ff45a526a9364c59bcaa2d6ed2e6c2e2 (127.19.194.3:46267): tablet 458ddeb03772404a8ce7ce0084990d04 (table test-table [id=7472995e6d5c455c90d88fb1e116f387]) successfully deleted
W20260806 18:38:15.446580 7270 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.19.194.1:35909 (error 108)
I20260806 18:38:15.447096 7272 catalog_manager.cc:4841] TS 80b05b9901c94886a0122923c8e393f7 (127.19.194.2:42143): tablet 458ddeb03772404a8ce7ce0084990d04 (table test-table [id=7472995e6d5c455c90d88fb1e116f387]) successfully deleted
I20260806 18:38:15.447630 20232 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJW69tn/build/debug/bin/kudu with pid 7488
W20260806 18:38:15.447654 7773 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 18:38:15.447294 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.194.1:35909 (local address 127.0.0.1:43384)
0806 18:38:15.447433 (+ 139us) negotiation.cc:107] Waiting for socket to connect
0806 18:38:15.447551 (+ 118us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.194.1:35909: connect: Connection reset by peer (error 104)
Metrics: {"client-negotiator.queue_time_us":44}
W20260806 18:38:15.449440 7270 catalog_manager.cc:4584] TS 50a5e173eeed4f9eaf2225222fae35b2 (127.19.194.1:35909): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 458ddeb03772404a8ce7ce0084990d04: Network error: Client connection negotiation failed: client connection to 127.19.194.1:35909: connect: Connection reset by peer (error 104)
I20260806 18:38:15.456645 20232 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJW69tn/build/debug/bin/kudu with pid 7621
I20260806 18:38:15.462782 20232 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJW69tn/build/debug/bin/kudu with pid 7253
2026-08-06T18:38:15Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/3 (1152 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/4
Loading random data
Initializing database '/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786041246094603-20232-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 18:38:15 dist-test-slave-1xrh krb5kdc[7808](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 18:38:15 dist-test-slave-1xrh krb5kdc[7808](info): set up 2 sockets
Aug 06 18:38:15 dist-test-slave-1xrh krb5kdc[7808](info): commencing operation
krb5kdc: starting...
W20260806 18:38:16.528172 20232 mini_kdc.cc:121] Time spent starting KDC: real 1.043s user 0.001s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786041246094603-20232-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786041246094603-20232-0/krb5kdc/test-user.keytab.
Aug 06 18:38:16 dist-test-slave-1xrh krb5kdc[7808](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041496, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T18:38:16Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T18:38:16Z Disabled control of system clock
WARNING: no policy specified for kudu/127.19.194.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.19.194.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.19.194.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786041246094603-20232-0/krb5kdc/kudu_127.19.194.62.keytab.
Entry for principal kudu/127.19.194.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786041246094603-20232-0/krb5kdc/kudu_127.19.194.62.keytab.
I20260806 18:38:16.650182 20232 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJW69tn/build/debug/bin/kudu
/tmp/dist-test-taskJW69tn/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786041246094603-20232-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786041246094603-20232-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786041246094603-20232-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786041246094603-20232-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.19.194.62:35573
--webserver_interface=127.19.194.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786041246094603-20232-0/krb5kdc/kudu_127.19.194.62.keytab
--principal=kudu/127.19.194.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.19.194.20:36013
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.19.194.62:35573
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786041246094603-20232-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786041246094603-20232-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786041246094603-20232-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 18:38:16.756336 7823 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 18:38:16.756625 7823 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 18:38:16.756704 7823 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 18:38:16.760613 7823 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 18:38:16.760689 7823 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 18:38:16.760730 7823 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 18:38:16.760798 7823 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 18:38:16.760835 7823 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 18:38:16.765969 7823 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.194.20:36013
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786041246094603-20232-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786041246094603-20232-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.19.194.62:35573
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786041246094603-20232-0/krb5kdc/kudu_127.19.194.62.keytab
--principal=kudu/127.19.194.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.19.194.62:35573
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786041246094603-20232-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.19.194.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7823
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786041246094603-20232-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 17:59:13 UTC on ccycloud.quasar-wvsnzx.root.comops.site
build id 4571028
I20260806 18:38:16.767006 7823 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 18:38:16.767994 7823 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 18:38:16.776100 7831 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:38:16.776144 7828 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:38:16.776129 7829 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 18:38:16.776288 7823 server_base.cc:1034] running on GCE node
I20260806 18:38:16.776937 7823 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 18:38:16.778034 7823 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 18:38:16.779193 7823 hybrid_clock.cc:648] HybridClock initialized: now 1786041496779170 us; error 29 us; skew 500 ppm
Aug 06 18:38:16 dist-test-slave-1xrh krb5kdc[7808](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041496, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.194.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 18:38:16.781869 7823 init.cc:377] Logged in from keytab as kudu/127.19.194.62@KRBTEST.COM (short username kudu)
I20260806 18:38:16.782933 7823 webserver.cc:460] Webserver started at http://127.19.194.62:43905/ using document root <none> and password file <none>
I20260806 18:38:16.783584 7823 fs_manager.cc:359] Metadata directory not provided
I20260806 18:38:16.783684 7823 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 18:38:16.783936 7823 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 18:38:16.786161 7823 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786041246094603-20232-0/minicluster-data/master-0/data/instance:
uuid: "e33e44a98e3c4f38b68948dd763b26f7"
format_stamp: "Formatted at 2026-08-06 18:38:16 on dist-test-slave-1xrh"
server_key: "2708943b1fbf98de13d0f1905b5a7671"
server_key_iv: "0e69ac17677b030b7482a0e663f0231d"
server_key_version: "encryptionkey@0"
I20260806 18:38:16.786696 7823 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786041246094603-20232-0/minicluster-data/master-0/wal/instance:
uuid: "e33e44a98e3c4f38b68948dd763b26f7"
format_stamp: "Formatted at 2026-08-06 18:38:16 on dist-test-slave-1xrh"
server_key: "2708943b1fbf98de13d0f1905b5a7671"
server_key_iv: "0e69ac17677b030b7482a0e663f0231d"
server_key_version: "encryptionkey@0"
I20260806 18:38:16.790758 7823 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260806 18:38:16.793116 7838 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 18:38:16.794269 7823 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260806 18:38:16.794397 7823 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786041246094603-20232-0/minicluster-data/master-0/data,/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786041246094603-20232-0/minicluster-data/master-0/wal
uuid: "e33e44a98e3c4f38b68948dd763b26f7"
format_stamp: "Formatted at 2026-08-06 18:38:16 on dist-test-slave-1xrh"
server_key: "2708943b1fbf98de13d0f1905b5a7671"
server_key_iv: "0e69ac17677b030b7482a0e663f0231d"
server_key_version: "encryptionkey@0"
I20260806 18:38:16.794504 7823 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786041246094603-20232-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786041246094603-20232-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786041246094603-20232-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 18:38:16.808540 7823 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 18:38:16.811678 7823 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 18:38:16.811869 7823 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 18:38:16.818655 7823 rpc_server.cc:307] RPC server started. Bound to: 127.19.194.62:35573
I20260806 18:38:16.818672 7892 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.194.62:35573 every 8 connection(s)
I20260806 18:38:16.819576 7823 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786041246094603-20232-0/minicluster-data/master-0/data/info.pb
I20260806 18:38:16.822571 7893 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 18:38:16.826010 20232 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJW69tn/build/debug/bin/kudu as pid 7823
I20260806 18:38:16.826153 20232 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786041246094603-20232-0/minicluster-data/master-0/wal/instance
I20260806 18:38:16.826469 20232 external_mini_cluster.cc:1428] Setting key 0d22be113595b2f439fadbba71705c5b
I20260806 18:38:16.828614 7893 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e33e44a98e3c4f38b68948dd763b26f7: Bootstrap starting.
I20260806 18:38:16.831076 7893 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P e33e44a98e3c4f38b68948dd763b26f7: Neither blocks nor log segments found. Creating new log.
I20260806 18:38:16.831885 7893 log.cc:824] T 00000000000000000000000000000000 P e33e44a98e3c4f38b68948dd763b26f7: Log is configured to *not* fsync() on all Append() calls
I20260806 18:38:16.833978 7893 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e33e44a98e3c4f38b68948dd763b26f7: No bootstrap required, opened a new log
Aug 06 18:38:16 dist-test-slave-1xrh krb5kdc[7808](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041496, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.19.194.62@KRBTEST.COM
I20260806 18:38:16.838526 7893 raft_consensus.cc:348] T 00000000000000000000000000000000 P e33e44a98e3c4f38b68948dd763b26f7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e33e44a98e3c4f38b68948dd763b26f7" member_type: VOTER last_known_addr { host: "127.19.194.62" port: 35573 } }
I20260806 18:38:16.838801 7893 raft_consensus.cc:374] T 00000000000000000000000000000000 P e33e44a98e3c4f38b68948dd763b26f7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 18:38:16.838872 7893 raft_consensus.cc:729] T 00000000000000000000000000000000 P e33e44a98e3c4f38b68948dd763b26f7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e33e44a98e3c4f38b68948dd763b26f7, State: Initialized, Role: FOLLOWER
I20260806 18:38:16.839371 7893 consensus_queue.cc:260] T 00000000000000000000000000000000 P e33e44a98e3c4f38b68948dd763b26f7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e33e44a98e3c4f38b68948dd763b26f7" member_type: VOTER last_known_addr { host: "127.19.194.62" port: 35573 } }
I20260806 18:38:16.839542 7893 raft_consensus.cc:388] T 00000000000000000000000000000000 P e33e44a98e3c4f38b68948dd763b26f7 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 18:38:16.839622 7893 raft_consensus.cc:482] T 00000000000000000000000000000000 P e33e44a98e3c4f38b68948dd763b26f7 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 18:38:16.839785 7893 raft_consensus.cc:3037] T 00000000000000000000000000000000 P e33e44a98e3c4f38b68948dd763b26f7 [term 0 FOLLOWER]: Advancing to term 1
I20260806 18:38:16.840714 7893 raft_consensus.cc:504] T 00000000000000000000000000000000 P e33e44a98e3c4f38b68948dd763b26f7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e33e44a98e3c4f38b68948dd763b26f7" member_type: VOTER last_known_addr { host: "127.19.194.62" port: 35573 } }
I20260806 18:38:16.841054 7896 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:38:16.827891 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44238 (local address 127.19.194.62:35573)
0806 18:38:16.828310 (+ 419us) server_negotiation.cc:206] Beginning negotiation
0806 18:38:16.828331 (+ 21us) server_negotiation.cc:399] Waiting for connection header
0806 18:38:16.828367 (+ 36us) server_negotiation.cc:407] Connection header received
0806 18:38:16.829024 (+ 657us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:38:16.829042 (+ 18us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:38:16.829352 (+ 310us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:38:16.829621 (+ 269us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:38:16.830556 (+ 935us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:38:16.831617 (+ 1061us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:38:16.832683 (+ 1066us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:38:16.833120 (+ 437us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:38:16.833234 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:38:16.835963 (+ 2729us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:38:16.835970 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:38:16.835985 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 18:38:16.836015 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:38:16.838006 (+ 1991us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:38:16.838439 (+ 433us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:38:16.838445 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:38:16.838450 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:38:16.838604 (+ 154us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:38:16.838892 (+ 288us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:38:16.838896 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:38:16.838897 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:38:16.839408 (+ 511us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:38:16.839606 (+ 198us) server_negotiation.cc:1035] Waiting for connection context
0806 18:38:16.840076 (+ 470us) server_negotiation.cc:299] Negotiation successful
0806 18:38:16.840314 (+ 238us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":275,"thread_start_us":136,"threads_started":1}
I20260806 18:38:16.841053 7893 leader_election.cc:302] T 00000000000000000000000000000000 P e33e44a98e3c4f38b68948dd763b26f7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: e33e44a98e3c4f38b68948dd763b26f7; no voters:
I20260806 18:38:16.841426 7893 leader_election.cc:288] T 00000000000000000000000000000000 P e33e44a98e3c4f38b68948dd763b26f7 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 18:38:16.841512 7898 raft_consensus.cc:2781] T 00000000000000000000000000000000 P e33e44a98e3c4f38b68948dd763b26f7 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 18:38:16.841701 7898 raft_consensus.cc:686] T 00000000000000000000000000000000 P e33e44a98e3c4f38b68948dd763b26f7 [term 1 LEADER]: Becoming Leader. State: Replica: e33e44a98e3c4f38b68948dd763b26f7, State: Running, Role: LEADER
I20260806 18:38:16.841988 7898 consensus_queue.cc:237] T 00000000000000000000000000000000 P e33e44a98e3c4f38b68948dd763b26f7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e33e44a98e3c4f38b68948dd763b26f7" member_type: VOTER last_known_addr { host: "127.19.194.62" port: 35573 } }
I20260806 18:38:16.842674 7893 sys_catalog.cc:564] T 00000000000000000000000000000000 P e33e44a98e3c4f38b68948dd763b26f7 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 18:38:16.844668 7900 sys_catalog.cc:455] T 00000000000000000000000000000000 P e33e44a98e3c4f38b68948dd763b26f7 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "e33e44a98e3c4f38b68948dd763b26f7" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e33e44a98e3c4f38b68948dd763b26f7" member_type: VOTER last_known_addr { host: "127.19.194.62" port: 35573 } } }
I20260806 18:38:16.844810 7900 sys_catalog.cc:458] T 00000000000000000000000000000000 P e33e44a98e3c4f38b68948dd763b26f7 [sys.catalog]: This master's current role is: LEADER
I20260806 18:38:16.845182 7899 sys_catalog.cc:455] T 00000000000000000000000000000000 P e33e44a98e3c4f38b68948dd763b26f7 [sys.catalog]: SysCatalogTable state changed. Reason: New leader e33e44a98e3c4f38b68948dd763b26f7. Latest consensus state: current_term: 1 leader_uuid: "e33e44a98e3c4f38b68948dd763b26f7" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e33e44a98e3c4f38b68948dd763b26f7" member_type: VOTER last_known_addr { host: "127.19.194.62" port: 35573 } } }
I20260806 18:38:16.845278 7899 sys_catalog.cc:458] T 00000000000000000000000000000000 P e33e44a98e3c4f38b68948dd763b26f7 [sys.catalog]: This master's current role is: LEADER
I20260806 18:38:16.845291 7914 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 18:38:16.847929 7914 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 18:38:16.853724 7914 catalog_manager.cc:1347] Generated new cluster ID: 5959722bb2744bc69dd627805ab4dc58
I20260806 18:38:16.853785 7914 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 18:38:16.860292 7914 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 18:38:16.861400 7914 catalog_manager.cc:1504] Loading token signing keys...
I20260806 18:38:16.865314 7914 catalog_manager.cc:5868] T 00000000000000000000000000000000 P e33e44a98e3c4f38b68948dd763b26f7: Generated new TSK 0
I20260806 18:38:16.866019 7914 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.19.194.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.19.194.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.19.194.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786041246094603-20232-0/krb5kdc/kudu_127.19.194.1.keytab.
Entry for principal kudu/127.19.194.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786041246094603-20232-0/krb5kdc/kudu_127.19.194.1.keytab.
I20260806 18:38:16.902706 20232 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJW69tn/build/debug/bin/kudu
/tmp/dist-test-taskJW69tn/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786041246094603-20232-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786041246094603-20232-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786041246094603-20232-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786041246094603-20232-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.19.194.1:0
--local_ip_for_outbound_sockets=127.19.194.1
--webserver_interface=127.19.194.1
--webserver_port=0
--tserver_master_addrs=127.19.194.62:35573
--keytab_file=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786041246094603-20232-0/krb5kdc/kudu_127.19.194.1.keytab
--principal=kudu/127.19.194.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.19.194.20:36013
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786041246094603-20232-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786041246094603-20232-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786041246094603-20232-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 18:38:17.013041 7921 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 18:38:17.013300 7921 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 18:38:17.013365 7921 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 18:38:17.017257 7921 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 18:38:17.017330 7921 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 18:38:17.017416 7921 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.19.194.1
I20260806 18:38:17.022585 7921 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.194.20:36013
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786041246094603-20232-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786041246094603-20232-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786041246094603-20232-0/krb5kdc/kudu_127.19.194.1.keytab
--principal=kudu/127.19.194.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.19.194.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786041246094603-20232-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.19.194.1
--webserver_port=0
--tserver_master_addrs=127.19.194.62:35573
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7921
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.19.194.1
--log_dir=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786041246094603-20232-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 17:59:13 UTC on ccycloud.quasar-wvsnzx.root.comops.site
build id 4571028
I20260806 18:38:17.023645 7921 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 18:38:17.024657 7921 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 18:38:17.033936 7926 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:38:17.033972 7927 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 18:38:17.034056 7921 server_base.cc:1034] running on GCE node
W20260806 18:38:17.033960 7929 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 18:38:17.034570 7921 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 18:38:17.035240 7921 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 18:38:17.036387 7921 hybrid_clock.cc:648] HybridClock initialized: now 1786041497036358 us; error 40 us; skew 500 ppm
Aug 06 18:38:17 dist-test-slave-1xrh krb5kdc[7808](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041497, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.194.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 18:38:17.039003 7921 init.cc:377] Logged in from keytab as kudu/127.19.194.1@KRBTEST.COM (short username kudu)
I20260806 18:38:17.039837 7921 webserver.cc:460] Webserver started at http://127.19.194.1:43627/ using document root <none> and password file <none>
I20260806 18:38:17.040331 7921 fs_manager.cc:359] Metadata directory not provided
I20260806 18:38:17.040409 7921 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 18:38:17.040630 7921 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 18:38:17.042585 7921 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786041246094603-20232-0/minicluster-data/ts-0/data/instance:
uuid: "c7ecd856cb89430a87628891e50723e3"
format_stamp: "Formatted at 2026-08-06 18:38:17 on dist-test-slave-1xrh"
server_key: "b133011d59eae860ba5d41a7e9e7a5eb"
server_key_iv: "6e664a4ad11fcd2a42135427fe6731a2"
server_key_version: "encryptionkey@0"
I20260806 18:38:17.043202 7921 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786041246094603-20232-0/minicluster-data/ts-0/wal/instance:
uuid: "c7ecd856cb89430a87628891e50723e3"
format_stamp: "Formatted at 2026-08-06 18:38:17 on dist-test-slave-1xrh"
server_key: "b133011d59eae860ba5d41a7e9e7a5eb"
server_key_iv: "6e664a4ad11fcd2a42135427fe6731a2"
server_key_version: "encryptionkey@0"
I20260806 18:38:17.046756 7921 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.000s sys 0.003s
I20260806 18:38:17.049103 7936 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 18:38:17.050239 7921 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260806 18:38:17.050366 7921 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786041246094603-20232-0/minicluster-data/ts-0/data,/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786041246094603-20232-0/minicluster-data/ts-0/wal
uuid: "c7ecd856cb89430a87628891e50723e3"
format_stamp: "Formatted at 2026-08-06 18:38:17 on dist-test-slave-1xrh"
server_key: "b133011d59eae860ba5d41a7e9e7a5eb"
server_key_iv: "6e664a4ad11fcd2a42135427fe6731a2"
server_key_version: "encryptionkey@0"
I20260806 18:38:17.050472 7921 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786041246094603-20232-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786041246094603-20232-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786041246094603-20232-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 18:38:17.062549 7921 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 18:38:17.065354 7921 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 18:38:17.065546 7921 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 18:38:17.066089 7921 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 18:38:17.066850 7921 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 18:38:17.066929 7921 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 18:38:17.066996 7921 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 18:38:17.067029 7921 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 18:38:17.075853 7921 rpc_server.cc:307] RPC server started. Bound to: 127.19.194.1:40991
I20260806 18:38:17.075874 8052 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.194.1:40991 every 8 connection(s)
I20260806 18:38:17.076639 7921 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786041246094603-20232-0/minicluster-data/ts-0/data/info.pb
I20260806 18:38:17.078586 20232 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJW69tn/build/debug/bin/kudu as pid 7921
I20260806 18:38:17.078719 20232 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786041246094603-20232-0/minicluster-data/ts-0/wal/instance
I20260806 18:38:17.079017 20232 external_mini_cluster.cc:1428] Setting key 9b192b3773c0c24a90776b8dc3cd8fc1
Aug 06 18:38:17 dist-test-slave-1xrh krb5kdc[7808](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041497, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.194.1@KRBTEST.COM for kudu/127.19.194.62@KRBTEST.COM
WARNING: no policy specified for kudu/127.19.194.2@KRBTEST.COM; defaulting to no policy
I20260806 18:38:17.091854 7896 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:38:17.078343 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.194.1:38263 (local address 127.19.194.62:35573)
0806 18:38:17.078506 (+ 163us) server_negotiation.cc:206] Beginning negotiation
0806 18:38:17.078512 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 18:38:17.079878 (+ 1366us) server_negotiation.cc:407] Connection header received
0806 18:38:17.080775 (+ 897us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:38:17.080781 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:38:17.080838 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:38:17.080931 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:38:17.081972 (+ 1041us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:38:17.082674 (+ 702us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:38:17.084475 (+ 1801us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:38:17.084961 (+ 486us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:38:17.085100 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:38:17.088278 (+ 3178us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:38:17.088283 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:38:17.088285 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 18:38:17.088312 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:38:17.089853 (+ 1541us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:38:17.090461 (+ 608us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:38:17.090463 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:38:17.090464 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:38:17.090540 (+ 76us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:38:17.090833 (+ 293us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:38:17.090835 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:38:17.090836 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:38:17.091021 (+ 185us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:38:17.091090 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0806 18:38:17.091626 (+ 536us) server_negotiation.cc:299] Negotiation successful
0806 18:38:17.091732 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260806 18:38:17.092626 8055 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:38:17.078722 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.194.62:35573 (local address 127.19.194.1:38263)
0806 18:38:17.079727 (+ 1005us) negotiation.cc:107] Waiting for socket to connect
0806 18:38:17.079778 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0806 18:38:17.080575 (+ 797us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:38:17.081077 (+ 502us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:38:17.081083 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:38:17.081402 (+ 319us) client_negotiation.cc:189] Negotiated authn=SASL
0806 18:38:17.081826 (+ 424us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:38:17.081838 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:38:17.082800 (+ 962us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:38:17.082805 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:38:17.084308 (+ 1503us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:38:17.084323 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:38:17.085182 (+ 859us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:38:17.085190 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:38:17.085441 (+ 251us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:38:17.086035 (+ 594us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 18:38:17.086066 (+ 31us) client_negotiation.cc:656] calling sasl_client_start()
0806 18:38:17.088109 (+ 2043us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 18:38:17.089972 (+ 1863us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:38:17.089980 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:38:17.089996 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:38:17.090346 (+ 350us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:38:17.090616 (+ 270us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:38:17.090618 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:38:17.090619 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:38:17.090736 (+ 117us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:38:17.091099 (+ 363us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 18:38:17.091104 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 18:38:17.091391 (+ 287us) client_negotiation.cc:769] Sending connection context
0806 18:38:17.091548 (+ 157us) client_negotiation.cc:240] Negotiation successful
0806 18:38:17.091773 (+ 225us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":804,"thread_start_us":646,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.19.194.2@KRBTEST.COM" created.
I20260806 18:38:17.093735 8053 heartbeater.cc:347] Connected to a master server at 127.19.194.62:35573
I20260806 18:38:17.093981 8053 heartbeater.cc:464] Registering TS with master...
I20260806 18:38:17.094529 8053 heartbeater.cc:511] Master 127.19.194.62:35573 requested a full tablet report, sending...
I20260806 18:38:17.096076 7856 ts_manager.cc:194] Registered new tserver with Master: c7ecd856cb89430a87628891e50723e3 (127.19.194.1:40991)
I20260806 18:38:17.097297 7856 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.19.194.1@KRBTEST.COM'} at 127.19.194.1:38263
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.19.194.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786041246094603-20232-0/krb5kdc/kudu_127.19.194.2.keytab.
Entry for principal kudu/127.19.194.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786041246094603-20232-0/krb5kdc/kudu_127.19.194.2.keytab.
I20260806 18:38:17.109689 20232 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJW69tn/build/debug/bin/kudu
/tmp/dist-test-taskJW69tn/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786041246094603-20232-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786041246094603-20232-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786041246094603-20232-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786041246094603-20232-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.19.194.2:0
--local_ip_for_outbound_sockets=127.19.194.2
--webserver_interface=127.19.194.2
--webserver_port=0
--tserver_master_addrs=127.19.194.62:35573
--keytab_file=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786041246094603-20232-0/krb5kdc/kudu_127.19.194.2.keytab
--principal=kudu/127.19.194.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.19.194.20:36013
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786041246094603-20232-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786041246094603-20232-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786041246094603-20232-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 18:38:17.218840 8058 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 18:38:17.219094 8058 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 18:38:17.219172 8058 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 18:38:17.223115 8058 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 18:38:17.223229 8058 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 18:38:17.223323 8058 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.19.194.2
I20260806 18:38:17.228474 8058 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.194.20:36013
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786041246094603-20232-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786041246094603-20232-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786041246094603-20232-0/krb5kdc/kudu_127.19.194.2.keytab
--principal=kudu/127.19.194.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.19.194.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786041246094603-20232-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.19.194.2
--webserver_port=0
--tserver_master_addrs=127.19.194.62:35573
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8058
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.19.194.2
--log_dir=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786041246094603-20232-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 17:59:13 UTC on ccycloud.quasar-wvsnzx.root.comops.site
build id 4571028
I20260806 18:38:17.229419 8058 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 18:38:17.230371 8058 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260806 18:38:17.239532 8058 server_base.cc:1034] running on GCE node
W20260806 18:38:17.239454 8063 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:38:17.239552 8066 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:38:17.239454 8064 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 18:38:17.240157 8058 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 18:38:17.240734 8058 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 18:38:17.241911 8058 hybrid_clock.cc:648] HybridClock initialized: now 1786041497241884 us; error 42 us; skew 500 ppm
Aug 06 18:38:17 dist-test-slave-1xrh krb5kdc[7808](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041497, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.194.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 18:38:17.244707 8058 init.cc:377] Logged in from keytab as kudu/127.19.194.2@KRBTEST.COM (short username kudu)
I20260806 18:38:17.245533 8058 webserver.cc:460] Webserver started at http://127.19.194.2:35509/ using document root <none> and password file <none>
I20260806 18:38:17.246052 8058 fs_manager.cc:359] Metadata directory not provided
I20260806 18:38:17.246140 8058 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 18:38:17.246353 8058 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 18:38:17.248297 8058 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786041246094603-20232-0/minicluster-data/ts-1/data/instance:
uuid: "89142e243e3246f88ba0a2f790685c55"
format_stamp: "Formatted at 2026-08-06 18:38:17 on dist-test-slave-1xrh"
server_key: "ce9b727da4f8b7224afe8aef33890139"
server_key_iv: "7fc3945101dead867e87d6e113b69a76"
server_key_version: "encryptionkey@0"
I20260806 18:38:17.248852 8058 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786041246094603-20232-0/minicluster-data/ts-1/wal/instance:
uuid: "89142e243e3246f88ba0a2f790685c55"
format_stamp: "Formatted at 2026-08-06 18:38:17 on dist-test-slave-1xrh"
server_key: "ce9b727da4f8b7224afe8aef33890139"
server_key_iv: "7fc3945101dead867e87d6e113b69a76"
server_key_version: "encryptionkey@0"
I20260806 18:38:17.252290 8058 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20260806 18:38:17.254674 8074 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 18:38:17.255867 8058 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260806 18:38:17.255959 8058 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786041246094603-20232-0/minicluster-data/ts-1/data,/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786041246094603-20232-0/minicluster-data/ts-1/wal
uuid: "89142e243e3246f88ba0a2f790685c55"
format_stamp: "Formatted at 2026-08-06 18:38:17 on dist-test-slave-1xrh"
server_key: "ce9b727da4f8b7224afe8aef33890139"
server_key_iv: "7fc3945101dead867e87d6e113b69a76"
server_key_version: "encryptionkey@0"
I20260806 18:38:17.256037 8058 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786041246094603-20232-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786041246094603-20232-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786041246094603-20232-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 18:38:17.281363 8058 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 18:38:17.284148 8058 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 18:38:17.284335 8058 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 18:38:17.284891 8058 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 18:38:17.285712 8058 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 18:38:17.285796 8058 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 18:38:17.285863 8058 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 18:38:17.285907 8058 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 18:38:17.294425 8058 rpc_server.cc:307] RPC server started. Bound to: 127.19.194.2:36937
I20260806 18:38:17.294505 8200 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.194.2:36937 every 8 connection(s)
I20260806 18:38:17.295305 8058 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786041246094603-20232-0/minicluster-data/ts-1/data/info.pb
I20260806 18:38:17.295799 20232 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJW69tn/build/debug/bin/kudu as pid 8058
I20260806 18:38:17.295924 20232 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786041246094603-20232-0/minicluster-data/ts-1/wal/instance
I20260806 18:38:17.296236 20232 external_mini_cluster.cc:1428] Setting key e4b158578ed29d0860d4a0c519a32b13
Aug 06 18:38:17 dist-test-slave-1xrh krb5kdc[7808](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041497, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.194.2@KRBTEST.COM for kudu/127.19.194.62@KRBTEST.COM
WARNING: no policy specified for kudu/127.19.194.3@KRBTEST.COM; defaulting to no policy
I20260806 18:38:17.309154 7896 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:38:17.297095 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.194.2:46203 (local address 127.19.194.62:35573)
0806 18:38:17.297247 (+ 152us) server_negotiation.cc:206] Beginning negotiation
0806 18:38:17.297252 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 18:38:17.298058 (+ 806us) server_negotiation.cc:407] Connection header received
0806 18:38:17.299530 (+ 1472us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:38:17.299533 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:38:17.299587 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:38:17.299671 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:38:17.300712 (+ 1041us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:38:17.301236 (+ 524us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:38:17.302366 (+ 1130us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:38:17.302678 (+ 312us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:38:17.302799 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:38:17.305390 (+ 2591us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:38:17.305394 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:38:17.305396 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 18:38:17.305425 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:38:17.306884 (+ 1459us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:38:17.307631 (+ 747us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:38:17.307634 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:38:17.307635 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:38:17.307711 (+ 76us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:38:17.308079 (+ 368us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:38:17.308083 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:38:17.308089 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:38:17.308258 (+ 169us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:38:17.308362 (+ 104us) server_negotiation.cc:1035] Waiting for connection context
0806 18:38:17.308873 (+ 511us) server_negotiation.cc:299] Negotiation successful
0806 18:38:17.308993 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.19.194.3@KRBTEST.COM" created.
I20260806 18:38:17.310030 8203 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:38:17.297455 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.194.62:35573 (local address 127.19.194.2:46203)
0806 18:38:17.297905 (+ 450us) negotiation.cc:107] Waiting for socket to connect
0806 18:38:17.297955 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0806 18:38:17.299301 (+ 1346us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:38:17.299821 (+ 520us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:38:17.299827 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:38:17.300148 (+ 321us) client_negotiation.cc:189] Negotiated authn=SASL
0806 18:38:17.300576 (+ 428us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:38:17.300588 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:38:17.301388 (+ 800us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:38:17.301391 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:38:17.302232 (+ 841us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:38:17.302242 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:38:17.302829 (+ 587us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:38:17.302833 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:38:17.302994 (+ 161us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:38:17.303573 (+ 579us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 18:38:17.303589 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0806 18:38:17.305214 (+ 1625us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 18:38:17.307008 (+ 1794us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:38:17.307013 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:38:17.307027 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:38:17.307355 (+ 328us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:38:17.307838 (+ 483us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:38:17.307841 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:38:17.307843 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:38:17.307976 (+ 133us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:38:17.308362 (+ 386us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 18:38:17.308369 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 18:38:17.308591 (+ 222us) client_negotiation.cc:769] Sending connection context
0806 18:38:17.308809 (+ 218us) client_negotiation.cc:240] Negotiation successful
0806 18:38:17.309101 (+ 292us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":260,"thread_start_us":137,"threads_started":1}
I20260806 18:38:17.315008 8201 heartbeater.cc:347] Connected to a master server at 127.19.194.62:35573
I20260806 18:38:17.315246 8201 heartbeater.cc:464] Registering TS with master...
I20260806 18:38:17.315798 8201 heartbeater.cc:511] Master 127.19.194.62:35573 requested a full tablet report, sending...
I20260806 18:38:17.316960 7856 ts_manager.cc:194] Registered new tserver with Master: 89142e243e3246f88ba0a2f790685c55 (127.19.194.2:36937)
I20260806 18:38:17.317487 7856 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.19.194.2@KRBTEST.COM'} at 127.19.194.2:46203
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.19.194.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786041246094603-20232-0/krb5kdc/kudu_127.19.194.3.keytab.
Entry for principal kudu/127.19.194.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786041246094603-20232-0/krb5kdc/kudu_127.19.194.3.keytab.
I20260806 18:38:17.326740 20232 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJW69tn/build/debug/bin/kudu
/tmp/dist-test-taskJW69tn/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786041246094603-20232-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786041246094603-20232-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786041246094603-20232-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786041246094603-20232-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.19.194.3:0
--local_ip_for_outbound_sockets=127.19.194.3
--webserver_interface=127.19.194.3
--webserver_port=0
--tserver_master_addrs=127.19.194.62:35573
--keytab_file=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786041246094603-20232-0/krb5kdc/kudu_127.19.194.3.keytab
--principal=kudu/127.19.194.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.19.194.20:36013
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786041246094603-20232-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786041246094603-20232-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786041246094603-20232-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 18:38:17.437224 8206 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 18:38:17.437474 8206 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 18:38:17.437551 8206 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 18:38:17.441573 8206 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 18:38:17.441644 8206 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 18:38:17.441726 8206 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.19.194.3
I20260806 18:38:17.446970 8206 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.194.20:36013
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786041246094603-20232-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786041246094603-20232-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786041246094603-20232-0/krb5kdc/kudu_127.19.194.3.keytab
--principal=kudu/127.19.194.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.19.194.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786041246094603-20232-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.19.194.3
--webserver_port=0
--tserver_master_addrs=127.19.194.62:35573
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8206
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.19.194.3
--log_dir=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786041246094603-20232-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 17:59:13 UTC on ccycloud.quasar-wvsnzx.root.comops.site
build id 4571028
I20260806 18:38:17.447935 8206 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 18:38:17.448884 8206 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 18:38:17.457784 8211 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:38:17.457794 8214 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:38:17.457810 8212 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 18:38:17.458009 8206 server_base.cc:1034] running on GCE node
I20260806 18:38:17.458457 8206 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 18:38:17.459021 8206 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 18:38:17.460189 8206 hybrid_clock.cc:648] HybridClock initialized: now 1786041497460176 us; error 23 us; skew 500 ppm
Aug 06 18:38:17 dist-test-slave-1xrh krb5kdc[7808](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041497, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.194.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 18:38:17.462924 8206 init.cc:377] Logged in from keytab as kudu/127.19.194.3@KRBTEST.COM (short username kudu)
I20260806 18:38:17.463797 8206 webserver.cc:460] Webserver started at http://127.19.194.3:41187/ using document root <none> and password file <none>
I20260806 18:38:17.464345 8206 fs_manager.cc:359] Metadata directory not provided
I20260806 18:38:17.464442 8206 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 18:38:17.464654 8206 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 18:38:17.466580 8206 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786041246094603-20232-0/minicluster-data/ts-2/data/instance:
uuid: "1b2a951cdee64398a62268135c5a612d"
format_stamp: "Formatted at 2026-08-06 18:38:17 on dist-test-slave-1xrh"
server_key: "5908a880dd68fcf90fa8c936a284d548"
server_key_iv: "b807bd69e04b43f00e64dc902687697c"
server_key_version: "encryptionkey@0"
I20260806 18:38:17.467190 8206 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786041246094603-20232-0/minicluster-data/ts-2/wal/instance:
uuid: "1b2a951cdee64398a62268135c5a612d"
format_stamp: "Formatted at 2026-08-06 18:38:17 on dist-test-slave-1xrh"
server_key: "5908a880dd68fcf90fa8c936a284d548"
server_key_iv: "b807bd69e04b43f00e64dc902687697c"
server_key_version: "encryptionkey@0"
I20260806 18:38:17.470669 8206 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.002s sys 0.001s
I20260806 18:38:17.473019 8222 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 18:38:17.474112 8206 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.000s sys 0.002s
I20260806 18:38:17.474236 8206 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786041246094603-20232-0/minicluster-data/ts-2/data,/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786041246094603-20232-0/minicluster-data/ts-2/wal
uuid: "1b2a951cdee64398a62268135c5a612d"
format_stamp: "Formatted at 2026-08-06 18:38:17 on dist-test-slave-1xrh"
server_key: "5908a880dd68fcf90fa8c936a284d548"
server_key_iv: "b807bd69e04b43f00e64dc902687697c"
server_key_version: "encryptionkey@0"
I20260806 18:38:17.474336 8206 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786041246094603-20232-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786041246094603-20232-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786041246094603-20232-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 18:38:17.489022 8206 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 18:38:17.491634 8206 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 18:38:17.491820 8206 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 18:38:17.492357 8206 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 18:38:17.493146 8206 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 18:38:17.493209 8206 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 18:38:17.493270 8206 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 18:38:17.493312 8206 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 18:38:17.502146 8206 rpc_server.cc:307] RPC server started. Bound to: 127.19.194.3:33701
I20260806 18:38:17.502187 8341 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.194.3:33701 every 8 connection(s)
I20260806 18:38:17.502978 8206 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786041246094603-20232-0/minicluster-data/ts-2/data/info.pb
Aug 06 18:38:17 dist-test-slave-1xrh krb5kdc[7808](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041497, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.194.3@KRBTEST.COM for kudu/127.19.194.62@KRBTEST.COM
I20260806 18:38:17.512992 20232 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJW69tn/build/debug/bin/kudu as pid 8206
I20260806 18:38:17.513130 20232 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786041246094603-20232-0/minicluster-data/ts-2/wal/instance
I20260806 18:38:17.513527 20232 external_mini_cluster.cc:1428] Setting key 732282aaf742d6d32582e31c88aeff62
I20260806 18:38:17.516271 7896 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:38:17.504694 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.194.3:45763 (local address 127.19.194.62:35573)
0806 18:38:17.504836 (+ 142us) server_negotiation.cc:206] Beginning negotiation
0806 18:38:17.504839 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 18:38:17.505630 (+ 791us) server_negotiation.cc:407] Connection header received
0806 18:38:17.506948 (+ 1318us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:38:17.506952 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:38:17.507031 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:38:17.507128 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:38:17.508292 (+ 1164us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:38:17.508828 (+ 536us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:38:17.509904 (+ 1076us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:38:17.510116 (+ 212us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:38:17.510219 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:38:17.512588 (+ 2369us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:38:17.512592 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:38:17.512594 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 18:38:17.512615 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:38:17.514096 (+ 1481us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:38:17.514607 (+ 511us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:38:17.514609 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:38:17.514611 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:38:17.514714 (+ 103us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:38:17.515142 (+ 428us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:38:17.515148 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:38:17.515151 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:38:17.515325 (+ 174us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:38:17.515429 (+ 104us) server_negotiation.cc:1035] Waiting for connection context
0806 18:38:17.515990 (+ 561us) server_negotiation.cc:299] Negotiation successful
0806 18:38:17.516106 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260806 18:38:17.516978 8344 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:38:17.505064 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.194.62:35573 (local address 127.19.194.3:45763)
0806 18:38:17.505491 (+ 427us) negotiation.cc:107] Waiting for socket to connect
0806 18:38:17.505531 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0806 18:38:17.506743 (+ 1212us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:38:17.507314 (+ 571us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:38:17.507321 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:38:17.507648 (+ 327us) client_negotiation.cc:189] Negotiated authn=SASL
0806 18:38:17.508147 (+ 499us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:38:17.508161 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:38:17.508956 (+ 795us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:38:17.508959 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:38:17.509776 (+ 817us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:38:17.509785 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:38:17.510254 (+ 469us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:38:17.510258 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:38:17.510417 (+ 159us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:38:17.510996 (+ 579us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 18:38:17.511012 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0806 18:38:17.512419 (+ 1407us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 18:38:17.514214 (+ 1795us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:38:17.514219 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:38:17.514232 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:38:17.514513 (+ 281us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:38:17.514851 (+ 338us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:38:17.514854 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:38:17.514856 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:38:17.515025 (+ 169us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:38:17.515429 (+ 404us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 18:38:17.515436 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 18:38:17.515747 (+ 311us) client_negotiation.cc:769] Sending connection context
0806 18:38:17.515946 (+ 199us) client_negotiation.cc:240] Negotiation successful
0806 18:38:17.516235 (+ 289us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":251,"thread_start_us":113,"threads_started":1}
I20260806 18:38:17.517897 8342 heartbeater.cc:347] Connected to a master server at 127.19.194.62:35573
I20260806 18:38:17.518152 8342 heartbeater.cc:464] Registering TS with master...
I20260806 18:38:17.518540 8342 heartbeater.cc:511] Master 127.19.194.62:35573 requested a full tablet report, sending...
I20260806 18:38:17.519436 7856 ts_manager.cc:194] Registered new tserver with Master: 1b2a951cdee64398a62268135c5a612d (127.19.194.3:33701)
I20260806 18:38:17.519948 7856 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.19.194.3@KRBTEST.COM'} at 127.19.194.3:45763
I20260806 18:38:17.527211 20232 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260806 18:38:17.535542 7896 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:38:17.529050 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44240 (local address 127.19.194.62:35573)
0806 18:38:17.529165 (+ 115us) server_negotiation.cc:206] Beginning negotiation
0806 18:38:17.529169 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 18:38:17.529334 (+ 165us) server_negotiation.cc:407] Connection header received
0806 18:38:17.529477 (+ 143us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:38:17.529480 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:38:17.529521 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:38:17.529593 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:38:17.530197 (+ 604us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:38:17.530689 (+ 492us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:38:17.531529 (+ 840us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:38:17.531736 (+ 207us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:38:17.531816 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:38:17.533000 (+ 1184us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:38:17.533003 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:38:17.533005 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 18:38:17.533027 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:38:17.534192 (+ 1165us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:38:17.534552 (+ 360us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:38:17.534554 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:38:17.534555 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:38:17.534624 (+ 69us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:38:17.534833 (+ 209us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:38:17.534835 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:38:17.534836 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:38:17.535020 (+ 184us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:38:17.535151 (+ 131us) server_negotiation.cc:1035] Waiting for connection context
0806 18:38:17.535342 (+ 191us) server_negotiation.cc:299] Negotiation successful
0806 18:38:17.535435 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260806 18:38:17.538012 7856 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:44240:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260806 18:38:17.541548 7856 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 18:38:17.555614 8353 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:38:17.549623 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.194.1:40991 (local address 127.0.0.1:38604)
0806 18:38:17.549938 (+ 315us) negotiation.cc:107] Waiting for socket to connect
0806 18:38:17.549980 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0806 18:38:17.550097 (+ 117us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:38:17.550875 (+ 778us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:38:17.550883 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:38:17.550925 (+ 42us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 18:38:17.551055 (+ 130us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:38:17.551061 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:38:17.552573 (+ 1512us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:38:17.552576 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:38:17.553907 (+ 1331us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:38:17.553917 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:38:17.555034 (+ 1117us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:38:17.555038 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:38:17.555182 (+ 144us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:38:17.555226 (+ 44us) client_negotiation.cc:769] Sending connection context
0806 18:38:17.555335 (+ 109us) client_negotiation.cc:240] Negotiation successful
0806 18:38:17.555449 (+ 114us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":152,"spinlock_wait_cycles":3328,"thread_start_us":86,"threads_started":1}
I20260806 18:38:17.556123 8355 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:38:17.550631 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.194.2:36937 (local address 127.0.0.1:54412)
0806 18:38:17.551512 (+ 881us) negotiation.cc:107] Waiting for socket to connect
0806 18:38:17.551525 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0806 18:38:17.551618 (+ 93us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:38:17.551963 (+ 345us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:38:17.551966 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:38:17.551985 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 18:38:17.552087 (+ 102us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:38:17.552094 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:38:17.553452 (+ 1358us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:38:17.553455 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:38:17.554634 (+ 1179us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:38:17.554644 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:38:17.555667 (+ 1023us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:38:17.555671 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:38:17.555751 (+ 80us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:38:17.555772 (+ 21us) client_negotiation.cc:769] Sending connection context
0806 18:38:17.555853 (+ 81us) client_negotiation.cc:240] Negotiation successful
0806 18:38:17.555971 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":819,"mutex_wait_us":73,"thread_start_us":87,"threads_started":1}
I20260806 18:38:17.556130 8358 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:38:17.551046 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54412 (local address 127.19.194.2:36937)
0806 18:38:17.551420 (+ 374us) server_negotiation.cc:206] Beginning negotiation
0806 18:38:17.551428 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 18:38:17.551604 (+ 176us) server_negotiation.cc:407] Connection header received
0806 18:38:17.551707 (+ 103us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:38:17.551710 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:38:17.551834 (+ 124us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:38:17.551982 (+ 148us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 18:38:17.552185 (+ 203us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:38:17.553300 (+ 1115us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:38:17.554754 (+ 1454us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:38:17.555542 (+ 788us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:38:17.555656 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:38:17.555810 (+ 154us) server_negotiation.cc:1035] Waiting for connection context
0806 18:38:17.555879 (+ 69us) server_negotiation.cc:299] Negotiation successful
0806 18:38:17.555975 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":278,"thread_start_us":164,"threads_started":1}
I20260806 18:38:17.556640 8356 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:38:17.550469 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54238 (local address 127.19.194.3:33701)
0806 18:38:17.551862 (+ 1393us) server_negotiation.cc:206] Beginning negotiation
0806 18:38:17.551874 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0806 18:38:17.551895 (+ 21us) server_negotiation.cc:407] Connection header received
0806 18:38:17.551963 (+ 68us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:38:17.551967 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:38:17.552122 (+ 155us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:38:17.552253 (+ 131us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 18:38:17.552452 (+ 199us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:38:17.553302 (+ 850us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:38:17.554782 (+ 1480us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:38:17.555574 (+ 792us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:38:17.555687 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:38:17.555839 (+ 152us) server_negotiation.cc:1035] Waiting for connection context
0806 18:38:17.556375 (+ 536us) server_negotiation.cc:299] Negotiation successful
0806 18:38:17.556489 (+ 114us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1293,"thread_start_us":95,"threads_started":1}
I20260806 18:38:17.556272 8357 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:38:17.550316 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.194.3:33701 (local address 127.0.0.1:54238)
0806 18:38:17.550837 (+ 521us) negotiation.cc:107] Waiting for socket to connect
0806 18:38:17.550852 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0806 18:38:17.550969 (+ 117us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:38:17.552244 (+ 1275us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:38:17.552247 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:38:17.552265 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 18:38:17.552340 (+ 75us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:38:17.552346 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:38:17.553461 (+ 1115us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:38:17.553465 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:38:17.554642 (+ 1177us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:38:17.554656 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:38:17.555913 (+ 1257us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:38:17.555916 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:38:17.556013 (+ 97us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:38:17.556033 (+ 20us) client_negotiation.cc:769] Sending connection context
0806 18:38:17.556098 (+ 65us) client_negotiation.cc:240] Negotiation successful
0806 18:38:17.556166 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":457,"mutex_wait_us":22,"thread_start_us":93,"threads_started":1}
I20260806 18:38:17.556806 8354 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:38:17.549759 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38604 (local address 127.19.194.1:40991)
0806 18:38:17.550017 (+ 258us) server_negotiation.cc:206] Beginning negotiation
0806 18:38:17.550036 (+ 19us) server_negotiation.cc:399] Waiting for connection header
0806 18:38:17.550058 (+ 22us) server_negotiation.cc:407] Connection header received
0806 18:38:17.550288 (+ 230us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:38:17.550294 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:38:17.550507 (+ 213us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:38:17.550667 (+ 160us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 18:38:17.551213 (+ 546us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:38:17.552364 (+ 1151us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:38:17.554034 (+ 1670us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:38:17.554919 (+ 885us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:38:17.555024 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:38:17.555215 (+ 191us) server_negotiation.cc:1035] Waiting for connection context
0806 18:38:17.556532 (+ 1317us) server_negotiation.cc:299] Negotiation successful
0806 18:38:17.556635 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":160,"thread_start_us":86,"threads_started":1}
I20260806 18:38:17.559338 7987 tablet_service.cc:1467] Processing CreateTablet for tablet a78933e030fe422aa0ac1673f1428e54 (DEFAULT_TABLE table=test-table [id=3ad7dd8098ef40c2b8787220c9820c1a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 18:38:17.559369 8275 tablet_service.cc:1467] Processing CreateTablet for tablet a78933e030fe422aa0ac1673f1428e54 (DEFAULT_TABLE table=test-table [id=3ad7dd8098ef40c2b8787220c9820c1a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 18:38:17.559361 8129 tablet_service.cc:1467] Processing CreateTablet for tablet a78933e030fe422aa0ac1673f1428e54 (DEFAULT_TABLE table=test-table [id=3ad7dd8098ef40c2b8787220c9820c1a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 18:38:17.560387 7987 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a78933e030fe422aa0ac1673f1428e54. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 18:38:17.560395 8129 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a78933e030fe422aa0ac1673f1428e54. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 18:38:17.560408 8275 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a78933e030fe422aa0ac1673f1428e54. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 18:38:17.566632 8359 tablet_bootstrap.cc:492] T a78933e030fe422aa0ac1673f1428e54 P c7ecd856cb89430a87628891e50723e3: Bootstrap starting.
I20260806 18:38:17.567981 8360 tablet_bootstrap.cc:492] T a78933e030fe422aa0ac1673f1428e54 P 89142e243e3246f88ba0a2f790685c55: Bootstrap starting.
I20260806 18:38:17.568823 8359 tablet_bootstrap.cc:654] T a78933e030fe422aa0ac1673f1428e54 P c7ecd856cb89430a87628891e50723e3: Neither blocks nor log segments found. Creating new log.
I20260806 18:38:17.569550 8360 tablet_bootstrap.cc:654] T a78933e030fe422aa0ac1673f1428e54 P 89142e243e3246f88ba0a2f790685c55: Neither blocks nor log segments found. Creating new log.
I20260806 18:38:17.569938 8359 log.cc:824] T a78933e030fe422aa0ac1673f1428e54 P c7ecd856cb89430a87628891e50723e3: Log is configured to *not* fsync() on all Append() calls
I20260806 18:38:17.570559 8361 tablet_bootstrap.cc:492] T a78933e030fe422aa0ac1673f1428e54 P 1b2a951cdee64398a62268135c5a612d: Bootstrap starting.
I20260806 18:38:17.570679 8360 log.cc:824] T a78933e030fe422aa0ac1673f1428e54 P 89142e243e3246f88ba0a2f790685c55: Log is configured to *not* fsync() on all Append() calls
I20260806 18:38:17.572089 8359 tablet_bootstrap.cc:492] T a78933e030fe422aa0ac1673f1428e54 P c7ecd856cb89430a87628891e50723e3: No bootstrap required, opened a new log
I20260806 18:38:17.572283 8359 ts_tablet_manager.cc:1397] T a78933e030fe422aa0ac1673f1428e54 P c7ecd856cb89430a87628891e50723e3: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260806 18:38:17.572517 8361 tablet_bootstrap.cc:654] T a78933e030fe422aa0ac1673f1428e54 P 1b2a951cdee64398a62268135c5a612d: Neither blocks nor log segments found. Creating new log.
I20260806 18:38:17.572791 8360 tablet_bootstrap.cc:492] T a78933e030fe422aa0ac1673f1428e54 P 89142e243e3246f88ba0a2f790685c55: No bootstrap required, opened a new log
I20260806 18:38:17.572959 8360 ts_tablet_manager.cc:1397] T a78933e030fe422aa0ac1673f1428e54 P 89142e243e3246f88ba0a2f790685c55: Time spent bootstrapping tablet: real 0.005s user 0.003s sys 0.000s
I20260806 18:38:17.573617 8361 log.cc:824] T a78933e030fe422aa0ac1673f1428e54 P 1b2a951cdee64398a62268135c5a612d: Log is configured to *not* fsync() on all Append() calls
I20260806 18:38:17.575178 8361 tablet_bootstrap.cc:492] T a78933e030fe422aa0ac1673f1428e54 P 1b2a951cdee64398a62268135c5a612d: No bootstrap required, opened a new log
I20260806 18:38:17.575358 8361 ts_tablet_manager.cc:1397] T a78933e030fe422aa0ac1673f1428e54 P 1b2a951cdee64398a62268135c5a612d: Time spent bootstrapping tablet: real 0.005s user 0.005s sys 0.000s
I20260806 18:38:17.576149 8359 raft_consensus.cc:348] T a78933e030fe422aa0ac1673f1428e54 P c7ecd856cb89430a87628891e50723e3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c7ecd856cb89430a87628891e50723e3" member_type: VOTER last_known_addr { host: "127.19.194.1" port: 40991 } } peers { permanent_uuid: "1b2a951cdee64398a62268135c5a612d" member_type: VOTER last_known_addr { host: "127.19.194.3" port: 33701 } } peers { permanent_uuid: "89142e243e3246f88ba0a2f790685c55" member_type: VOTER last_known_addr { host: "127.19.194.2" port: 36937 } }
I20260806 18:38:17.576418 8359 raft_consensus.cc:374] T a78933e030fe422aa0ac1673f1428e54 P c7ecd856cb89430a87628891e50723e3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 18:38:17.576476 8359 raft_consensus.cc:729] T a78933e030fe422aa0ac1673f1428e54 P c7ecd856cb89430a87628891e50723e3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c7ecd856cb89430a87628891e50723e3, State: Initialized, Role: FOLLOWER
I20260806 18:38:17.576639 8360 raft_consensus.cc:348] T a78933e030fe422aa0ac1673f1428e54 P 89142e243e3246f88ba0a2f790685c55 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c7ecd856cb89430a87628891e50723e3" member_type: VOTER last_known_addr { host: "127.19.194.1" port: 40991 } } peers { permanent_uuid: "1b2a951cdee64398a62268135c5a612d" member_type: VOTER last_known_addr { host: "127.19.194.3" port: 33701 } } peers { permanent_uuid: "89142e243e3246f88ba0a2f790685c55" member_type: VOTER last_known_addr { host: "127.19.194.2" port: 36937 } }
I20260806 18:38:17.576900 8360 raft_consensus.cc:374] T a78933e030fe422aa0ac1673f1428e54 P 89142e243e3246f88ba0a2f790685c55 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 18:38:17.576965 8360 raft_consensus.cc:729] T a78933e030fe422aa0ac1673f1428e54 P 89142e243e3246f88ba0a2f790685c55 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 89142e243e3246f88ba0a2f790685c55, State: Initialized, Role: FOLLOWER
I20260806 18:38:17.576931 8359 consensus_queue.cc:260] T a78933e030fe422aa0ac1673f1428e54 P c7ecd856cb89430a87628891e50723e3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c7ecd856cb89430a87628891e50723e3" member_type: VOTER last_known_addr { host: "127.19.194.1" port: 40991 } } peers { permanent_uuid: "1b2a951cdee64398a62268135c5a612d" member_type: VOTER last_known_addr { host: "127.19.194.3" port: 33701 } } peers { permanent_uuid: "89142e243e3246f88ba0a2f790685c55" member_type: VOTER last_known_addr { host: "127.19.194.2" port: 36937 } }
I20260806 18:38:17.577430 8360 consensus_queue.cc:260] T a78933e030fe422aa0ac1673f1428e54 P 89142e243e3246f88ba0a2f790685c55 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c7ecd856cb89430a87628891e50723e3" member_type: VOTER last_known_addr { host: "127.19.194.1" port: 40991 } } peers { permanent_uuid: "1b2a951cdee64398a62268135c5a612d" member_type: VOTER last_known_addr { host: "127.19.194.3" port: 33701 } } peers { permanent_uuid: "89142e243e3246f88ba0a2f790685c55" member_type: VOTER last_known_addr { host: "127.19.194.2" port: 36937 } }
I20260806 18:38:17.578012 8359 ts_tablet_manager.cc:1428] T a78933e030fe422aa0ac1673f1428e54 P c7ecd856cb89430a87628891e50723e3: Time spent starting tablet: real 0.006s user 0.007s sys 0.000s
I20260806 18:38:17.578351 8360 ts_tablet_manager.cc:1428] T a78933e030fe422aa0ac1673f1428e54 P 89142e243e3246f88ba0a2f790685c55: Time spent starting tablet: real 0.005s user 0.006s sys 0.000s
I20260806 18:38:17.578723 8201 heartbeater.cc:503] Master 127.19.194.62:35573 was elected leader, sending a full tablet report...
I20260806 18:38:17.578930 8053 heartbeater.cc:503] Master 127.19.194.62:35573 was elected leader, sending a full tablet report...
I20260806 18:38:17.579434 8361 raft_consensus.cc:348] T a78933e030fe422aa0ac1673f1428e54 P 1b2a951cdee64398a62268135c5a612d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c7ecd856cb89430a87628891e50723e3" member_type: VOTER last_known_addr { host: "127.19.194.1" port: 40991 } } peers { permanent_uuid: "1b2a951cdee64398a62268135c5a612d" member_type: VOTER last_known_addr { host: "127.19.194.3" port: 33701 } } peers { permanent_uuid: "89142e243e3246f88ba0a2f790685c55" member_type: VOTER last_known_addr { host: "127.19.194.2" port: 36937 } }
I20260806 18:38:17.579691 8361 raft_consensus.cc:374] T a78933e030fe422aa0ac1673f1428e54 P 1b2a951cdee64398a62268135c5a612d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 18:38:17.579733 8361 raft_consensus.cc:729] T a78933e030fe422aa0ac1673f1428e54 P 1b2a951cdee64398a62268135c5a612d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1b2a951cdee64398a62268135c5a612d, State: Initialized, Role: FOLLOWER
I20260806 18:38:17.580128 8361 consensus_queue.cc:260] T a78933e030fe422aa0ac1673f1428e54 P 1b2a951cdee64398a62268135c5a612d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c7ecd856cb89430a87628891e50723e3" member_type: VOTER last_known_addr { host: "127.19.194.1" port: 40991 } } peers { permanent_uuid: "1b2a951cdee64398a62268135c5a612d" member_type: VOTER last_known_addr { host: "127.19.194.3" port: 33701 } } peers { permanent_uuid: "89142e243e3246f88ba0a2f790685c55" member_type: VOTER last_known_addr { host: "127.19.194.2" port: 36937 } }
I20260806 18:38:17.580719 8342 heartbeater.cc:503] Master 127.19.194.62:35573 was elected leader, sending a full tablet report...
I20260806 18:38:17.581063 8361 ts_tablet_manager.cc:1428] T a78933e030fe422aa0ac1673f1428e54 P 1b2a951cdee64398a62268135c5a612d: Time spent starting tablet: real 0.006s user 0.005s sys 0.000s
I20260806 18:38:17.605424 8367 raft_consensus.cc:482] T a78933e030fe422aa0ac1673f1428e54 P 1b2a951cdee64398a62268135c5a612d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 18:38:17.605597 8367 raft_consensus.cc:504] T a78933e030fe422aa0ac1673f1428e54 P 1b2a951cdee64398a62268135c5a612d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c7ecd856cb89430a87628891e50723e3" member_type: VOTER last_known_addr { host: "127.19.194.1" port: 40991 } } peers { permanent_uuid: "1b2a951cdee64398a62268135c5a612d" member_type: VOTER last_known_addr { host: "127.19.194.3" port: 33701 } } peers { permanent_uuid: "89142e243e3246f88ba0a2f790685c55" member_type: VOTER last_known_addr { host: "127.19.194.2" port: 36937 } }
I20260806 18:38:17.606582 8367 leader_election.cc:288] T a78933e030fe422aa0ac1673f1428e54 P 1b2a951cdee64398a62268135c5a612d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c7ecd856cb89430a87628891e50723e3 (127.19.194.1:40991), 89142e243e3246f88ba0a2f790685c55 (127.19.194.2:36937)
I20260806 18:38:17.610714 8344 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:38:17.606830 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.194.1:40991 (local address 127.19.194.3:55837)
0806 18:38:17.606983 (+ 153us) negotiation.cc:107] Waiting for socket to connect
0806 18:38:17.607002 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0806 18:38:17.607108 (+ 106us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:38:17.607425 (+ 317us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:38:17.607429 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:38:17.607451 (+ 22us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 18:38:17.607601 (+ 150us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:38:17.607612 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:38:17.608583 (+ 971us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:38:17.608586 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:38:17.609619 (+ 1033us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:38:17.609631 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:38:17.610281 (+ 650us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:38:17.610285 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:38:17.610388 (+ 103us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:38:17.610428 (+ 40us) client_negotiation.cc:769] Sending connection context
0806 18:38:17.610502 (+ 74us) client_negotiation.cc:240] Negotiation successful
0806 18:38:17.610573 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":50,"mutex_wait_us":41}
I20260806 18:38:17.610745 8354 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:38:17.606926 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.194.3:55837 (local address 127.19.194.1:40991)
0806 18:38:17.607076 (+ 150us) server_negotiation.cc:206] Beginning negotiation
0806 18:38:17.607081 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 18:38:17.607098 (+ 17us) server_negotiation.cc:407] Connection header received
0806 18:38:17.607252 (+ 154us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:38:17.607256 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:38:17.607324 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:38:17.607414 (+ 90us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 18:38:17.607734 (+ 320us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:38:17.608473 (+ 739us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:38:17.609762 (+ 1289us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:38:17.610168 (+ 406us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:38:17.610258 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:38:17.610296 (+ 38us) server_negotiation.cc:1035] Waiting for connection context
0806 18:38:17.610530 (+ 234us) server_negotiation.cc:299] Negotiation successful
0806 18:38:17.610607 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260806 18:38:17.610975 8368 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:38:17.607114 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.194.2:36937 (local address 127.19.194.3:45727)
0806 18:38:17.607494 (+ 380us) negotiation.cc:107] Waiting for socket to connect
0806 18:38:17.607510 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0806 18:38:17.607597 (+ 87us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:38:17.607836 (+ 239us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:38:17.607839 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:38:17.607856 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 18:38:17.607997 (+ 141us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:38:17.608003 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:38:17.608745 (+ 742us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:38:17.608748 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:38:17.609766 (+ 1018us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:38:17.609777 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:38:17.610514 (+ 737us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:38:17.610519 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:38:17.610627 (+ 108us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:38:17.610648 (+ 21us) client_negotiation.cc:769] Sending connection context
0806 18:38:17.610722 (+ 74us) client_negotiation.cc:240] Negotiation successful
0806 18:38:17.610796 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":284,"mutex_wait_us":17,"thread_start_us":124,"threads_started":1}
I20260806 18:38:17.610998 8358 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:38:17.607274 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.194.3:45727 (local address 127.19.194.2:36937)
0806 18:38:17.607485 (+ 211us) server_negotiation.cc:206] Beginning negotiation
0806 18:38:17.607489 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 18:38:17.607594 (+ 105us) server_negotiation.cc:407] Connection header received
0806 18:38:17.607676 (+ 82us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:38:17.607680 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:38:17.607736 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:38:17.607801 (+ 65us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 18:38:17.608116 (+ 315us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:38:17.608637 (+ 521us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:38:17.609880 (+ 1243us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:38:17.610366 (+ 486us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:38:17.610505 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:38:17.610550 (+ 45us) server_negotiation.cc:1035] Waiting for connection context
0806 18:38:17.610765 (+ 215us) server_negotiation.cc:299] Negotiation successful
0806 18:38:17.610852 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":123}
I20260806 18:38:17.611706 8007 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a78933e030fe422aa0ac1673f1428e54" candidate_uuid: "1b2a951cdee64398a62268135c5a612d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c7ecd856cb89430a87628891e50723e3" is_pre_election: true
I20260806 18:38:17.611706 8152 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a78933e030fe422aa0ac1673f1428e54" candidate_uuid: "1b2a951cdee64398a62268135c5a612d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "89142e243e3246f88ba0a2f790685c55" is_pre_election: true
I20260806 18:38:17.611976 8152 raft_consensus.cc:2454] T a78933e030fe422aa0ac1673f1428e54 P 89142e243e3246f88ba0a2f790685c55 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1b2a951cdee64398a62268135c5a612d in term 0.
I20260806 18:38:17.611990 8007 raft_consensus.cc:2454] T a78933e030fe422aa0ac1673f1428e54 P c7ecd856cb89430a87628891e50723e3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1b2a951cdee64398a62268135c5a612d in term 0.
I20260806 18:38:17.612421 8224 leader_election.cc:302] T a78933e030fe422aa0ac1673f1428e54 P 1b2a951cdee64398a62268135c5a612d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1b2a951cdee64398a62268135c5a612d, 89142e243e3246f88ba0a2f790685c55; no voters:
I20260806 18:38:17.612679 8367 raft_consensus.cc:2781] T a78933e030fe422aa0ac1673f1428e54 P 1b2a951cdee64398a62268135c5a612d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 18:38:17.612767 8367 raft_consensus.cc:482] T a78933e030fe422aa0ac1673f1428e54 P 1b2a951cdee64398a62268135c5a612d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 18:38:17.612820 8367 raft_consensus.cc:3037] T a78933e030fe422aa0ac1673f1428e54 P 1b2a951cdee64398a62268135c5a612d [term 0 FOLLOWER]: Advancing to term 1
I20260806 18:38:17.613811 8367 raft_consensus.cc:504] T a78933e030fe422aa0ac1673f1428e54 P 1b2a951cdee64398a62268135c5a612d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c7ecd856cb89430a87628891e50723e3" member_type: VOTER last_known_addr { host: "127.19.194.1" port: 40991 } } peers { permanent_uuid: "1b2a951cdee64398a62268135c5a612d" member_type: VOTER last_known_addr { host: "127.19.194.3" port: 33701 } } peers { permanent_uuid: "89142e243e3246f88ba0a2f790685c55" member_type: VOTER last_known_addr { host: "127.19.194.2" port: 36937 } }
I20260806 18:38:17.614120 8367 leader_election.cc:288] T a78933e030fe422aa0ac1673f1428e54 P 1b2a951cdee64398a62268135c5a612d [CANDIDATE]: Term 1 election: Requested vote from peers c7ecd856cb89430a87628891e50723e3 (127.19.194.1:40991), 89142e243e3246f88ba0a2f790685c55 (127.19.194.2:36937)
I20260806 18:38:17.614436 8007 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a78933e030fe422aa0ac1673f1428e54" candidate_uuid: "1b2a951cdee64398a62268135c5a612d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c7ecd856cb89430a87628891e50723e3"
I20260806 18:38:17.614477 8152 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a78933e030fe422aa0ac1673f1428e54" candidate_uuid: "1b2a951cdee64398a62268135c5a612d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "89142e243e3246f88ba0a2f790685c55"
I20260806 18:38:17.614547 8007 raft_consensus.cc:3037] T a78933e030fe422aa0ac1673f1428e54 P c7ecd856cb89430a87628891e50723e3 [term 0 FOLLOWER]: Advancing to term 1
I20260806 18:38:17.614598 8152 raft_consensus.cc:3037] T a78933e030fe422aa0ac1673f1428e54 P 89142e243e3246f88ba0a2f790685c55 [term 0 FOLLOWER]: Advancing to term 1
I20260806 18:38:17.615628 8007 raft_consensus.cc:2454] T a78933e030fe422aa0ac1673f1428e54 P c7ecd856cb89430a87628891e50723e3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1b2a951cdee64398a62268135c5a612d in term 1.
I20260806 18:38:17.615772 8152 raft_consensus.cc:2454] T a78933e030fe422aa0ac1673f1428e54 P 89142e243e3246f88ba0a2f790685c55 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1b2a951cdee64398a62268135c5a612d in term 1.
I20260806 18:38:17.615962 8224 leader_election.cc:302] T a78933e030fe422aa0ac1673f1428e54 P 1b2a951cdee64398a62268135c5a612d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1b2a951cdee64398a62268135c5a612d, c7ecd856cb89430a87628891e50723e3; no voters:
I20260806 18:38:17.616125 8367 raft_consensus.cc:2781] T a78933e030fe422aa0ac1673f1428e54 P 1b2a951cdee64398a62268135c5a612d [term 1 FOLLOWER]: Leader election won for term 1
I20260806 18:38:17.616370 8367 raft_consensus.cc:686] T a78933e030fe422aa0ac1673f1428e54 P 1b2a951cdee64398a62268135c5a612d [term 1 LEADER]: Becoming Leader. State: Replica: 1b2a951cdee64398a62268135c5a612d, State: Running, Role: LEADER
I20260806 18:38:17.616721 8367 consensus_queue.cc:237] T a78933e030fe422aa0ac1673f1428e54 P 1b2a951cdee64398a62268135c5a612d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c7ecd856cb89430a87628891e50723e3" member_type: VOTER last_known_addr { host: "127.19.194.1" port: 40991 } } peers { permanent_uuid: "1b2a951cdee64398a62268135c5a612d" member_type: VOTER last_known_addr { host: "127.19.194.3" port: 33701 } } peers { permanent_uuid: "89142e243e3246f88ba0a2f790685c55" member_type: VOTER last_known_addr { host: "127.19.194.2" port: 36937 } }
I20260806 18:38:17.619650 7855 catalog_manager.cc:5495] T a78933e030fe422aa0ac1673f1428e54 P 1b2a951cdee64398a62268135c5a612d reported cstate change: term changed from 0 to 1, leader changed from <none> to 1b2a951cdee64398a62268135c5a612d (127.19.194.3). New cstate: current_term: 1 leader_uuid: "1b2a951cdee64398a62268135c5a612d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c7ecd856cb89430a87628891e50723e3" member_type: VOTER last_known_addr { host: "127.19.194.1" port: 40991 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1b2a951cdee64398a62268135c5a612d" member_type: VOTER last_known_addr { host: "127.19.194.3" port: 33701 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "89142e243e3246f88ba0a2f790685c55" member_type: VOTER last_known_addr { host: "127.19.194.2" port: 36937 } health_report { overall_health: UNKNOWN } } }
I20260806 18:38:17.632305 8356 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:38:17.629221 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54252 (local address 127.19.194.3:33701)
0806 18:38:17.629334 (+ 113us) server_negotiation.cc:206] Beginning negotiation
0806 18:38:17.629337 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 18:38:17.629349 (+ 12us) server_negotiation.cc:407] Connection header received
0806 18:38:17.629473 (+ 124us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:38:17.629475 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:38:17.629520 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:38:17.629600 (+ 80us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 18:38:17.629811 (+ 211us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:38:17.630278 (+ 467us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:38:17.631073 (+ 795us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:38:17.631317 (+ 244us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:38:17.631391 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:38:17.631642 (+ 251us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 18:38:17.632014 (+ 372us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 18:38:17.632098 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0806 18:38:17.632148 (+ 50us) server_negotiation.cc:299] Negotiation successful
0806 18:38:17.632189 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260806 18:38:17.637481 8152 raft_consensus.cc:1261] T a78933e030fe422aa0ac1673f1428e54 P 89142e243e3246f88ba0a2f790685c55 [term 1 FOLLOWER]: Refusing update from remote peer 1b2a951cdee64398a62268135c5a612d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 18:38:17.637497 8007 raft_consensus.cc:1261] T a78933e030fe422aa0ac1673f1428e54 P c7ecd856cb89430a87628891e50723e3 [term 1 FOLLOWER]: Refusing update from remote peer 1b2a951cdee64398a62268135c5a612d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 18:38:17.638072 8367 consensus_queue.cc:1035] T a78933e030fe422aa0ac1673f1428e54 P 1b2a951cdee64398a62268135c5a612d [LEADER]: Connected to new peer: Peer: permanent_uuid: "89142e243e3246f88ba0a2f790685c55" member_type: VOTER last_known_addr { host: "127.19.194.2" port: 36937 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 18:38:17.638255 8369 consensus_queue.cc:1035] T a78933e030fe422aa0ac1673f1428e54 P 1b2a951cdee64398a62268135c5a612d [LEADER]: Connected to new peer: Peer: permanent_uuid: "c7ecd856cb89430a87628891e50723e3" member_type: VOTER last_known_addr { host: "127.19.194.1" port: 40991 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 18:38:17.645452 8374 mvcc.cc:205] Tried to move back new op lower bound from 7315625974316773376 to 7315625974240784384. Current Snapshot: MvccSnapshot[applied={T|T < 7315625974316773376}]
I20260806 18:38:17.654206 7855 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:44240:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 18:38:17.654381 7855 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:44240:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 18:38:17.656777 7855 catalog_manager.cc:5782] T 00000000000000000000000000000000 P e33e44a98e3c4f38b68948dd763b26f7: Sending DeleteTablet for 3 replicas of tablet a78933e030fe422aa0ac1673f1428e54
I20260806 18:38:17.657555 8275 tablet_service.cc:1514] Processing DeleteTablet for tablet a78933e030fe422aa0ac1673f1428e54 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 18:38:17 UTC) from {username='kudu', principal='kudu/127.19.194.62@KRBTEST.COM'} at 127.0.0.1:54238
I20260806 18:38:17.657598 7987 tablet_service.cc:1514] Processing DeleteTablet for tablet a78933e030fe422aa0ac1673f1428e54 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 18:38:17 UTC) from {username='kudu', principal='kudu/127.19.194.62@KRBTEST.COM'} at 127.0.0.1:38604
I20260806 18:38:17.657626 8129 tablet_service.cc:1514] Processing DeleteTablet for tablet a78933e030fe422aa0ac1673f1428e54 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 18:38:17 UTC) from {username='kudu', principal='kudu/127.19.194.62@KRBTEST.COM'} at 127.0.0.1:54412
I20260806 18:38:17.658161 8384 tablet_replica.cc:330] T a78933e030fe422aa0ac1673f1428e54 P c7ecd856cb89430a87628891e50723e3: stopping tablet replica
I20260806 18:38:17.658495 20232 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJW69tn/build/debug/bin/kudu with pid 7921
I20260806 18:38:17.658478 8384 raft_consensus.cc:2229] T a78933e030fe422aa0ac1673f1428e54 P c7ecd856cb89430a87628891e50723e3 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 18:38:17.658782 8384 raft_consensus.cc:2258] T a78933e030fe422aa0ac1673f1428e54 P c7ecd856cb89430a87628891e50723e3 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 18:38:17.663303 8383 tablet_replica.cc:330] T a78933e030fe422aa0ac1673f1428e54 P 89142e243e3246f88ba0a2f790685c55: stopping tablet replica
I20260806 18:38:17.663589 8382 tablet_replica.cc:330] T a78933e030fe422aa0ac1673f1428e54 P 1b2a951cdee64398a62268135c5a612d: stopping tablet replica
I20260806 18:38:17.663628 8383 raft_consensus.cc:2229] T a78933e030fe422aa0ac1673f1428e54 P 89142e243e3246f88ba0a2f790685c55 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 18:38:17.663906 8382 raft_consensus.cc:2229] T a78933e030fe422aa0ac1673f1428e54 P 1b2a951cdee64398a62268135c5a612d [term 1 LEADER]: Raft consensus shutting down.
I20260806 18:38:17.664110 8383 raft_consensus.cc:2258] T a78933e030fe422aa0ac1673f1428e54 P 89142e243e3246f88ba0a2f790685c55 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 18:38:17.664283 8382 raft_consensus.cc:2258] T a78933e030fe422aa0ac1673f1428e54 P 1b2a951cdee64398a62268135c5a612d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 18:38:17.665796 8382 ts_tablet_manager.cc:1905] T a78933e030fe422aa0ac1673f1428e54 P 1b2a951cdee64398a62268135c5a612d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 18:38:17.665808 8383 ts_tablet_manager.cc:1905] T a78933e030fe422aa0ac1673f1428e54 P 89142e243e3246f88ba0a2f790685c55: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 18:38:17.668560 8383 ts_tablet_manager.cc:1918] T a78933e030fe422aa0ac1673f1428e54 P 89142e243e3246f88ba0a2f790685c55: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 18:38:17.668715 8383 log.cc:1196] T a78933e030fe422aa0ac1673f1428e54 P 89142e243e3246f88ba0a2f790685c55: Deleting WAL directory at /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786041246094603-20232-0/minicluster-data/ts-1/wal/wals/a78933e030fe422aa0ac1673f1428e54
I20260806 18:38:17.669101 8383 ts_tablet_manager.cc:1939] T a78933e030fe422aa0ac1673f1428e54 P 89142e243e3246f88ba0a2f790685c55: Deleting consensus metadata
I20260806 18:38:17.669324 8382 ts_tablet_manager.cc:1918] T a78933e030fe422aa0ac1673f1428e54 P 1b2a951cdee64398a62268135c5a612d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 18:38:17.669443 8382 log.cc:1196] T a78933e030fe422aa0ac1673f1428e54 P 1b2a951cdee64398a62268135c5a612d: Deleting WAL directory at /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786041246094603-20232-0/minicluster-data/ts-2/wal/wals/a78933e030fe422aa0ac1673f1428e54
W20260806 18:38:17.669652 7840 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.19.194.1:40991 (error 108)
I20260806 18:38:17.669790 8382 ts_tablet_manager.cc:1939] T a78933e030fe422aa0ac1673f1428e54 P 1b2a951cdee64398a62268135c5a612d: Deleting consensus metadata
I20260806 18:38:17.670471 20232 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJW69tn/build/debug/bin/kudu with pid 8058
I20260806 18:38:17.672080 7840 catalog_manager.cc:4841] TS 89142e243e3246f88ba0a2f790685c55 (127.19.194.2:36937): tablet a78933e030fe422aa0ac1673f1428e54 (table test-table [id=3ad7dd8098ef40c2b8787220c9820c1a]) successfully deleted
I20260806 18:38:17.675417 7840 catalog_manager.cc:4841] TS 1b2a951cdee64398a62268135c5a612d (127.19.194.3:33701): tablet a78933e030fe422aa0ac1673f1428e54 (table test-table [id=3ad7dd8098ef40c2b8787220c9820c1a]) successfully deleted
I20260806 18:38:17.676245 8357 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:38:17.675885 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.194.1:40991 (local address 127.0.0.1:38620)
0806 18:38:17.676068 (+ 183us) negotiation.cc:107] Waiting for socket to connect
0806 18:38:17.676147 (+ 79us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.194.1:40991: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":66}
W20260806 18:38:17.676484 7840 catalog_manager.cc:4584] TS c7ecd856cb89430a87628891e50723e3 (127.19.194.1:40991): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet a78933e030fe422aa0ac1673f1428e54: Network error: Client connection negotiation failed: client connection to 127.19.194.1:40991: connect: Connection refused (error 111)
I20260806 18:38:17.677807 20232 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJW69tn/build/debug/bin/kudu with pid 8206
I20260806 18:38:17.684002 20232 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJW69tn/build/debug/bin/kudu with pid 7823
2026-08-06T18:38:17Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/4 (2221 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/5
2026-08-06T18:38:17Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T18:38:17Z Disabled control of system clock
I20260806 18:38:17.715075 20232 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJW69tn/build/debug/bin/kudu
/tmp/dist-test-taskJW69tn/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786041246094603-20232-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786041246094603-20232-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786041246094603-20232-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786041246094603-20232-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.19.194.62:46231
--webserver_interface=127.19.194.62
--webserver_port=0
--builtin_ntp_servers=127.19.194.20:41559
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.19.194.62:46231
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260806 18:38:17.824463 8390 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 18:38:17.824720 8390 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 18:38:17.824786 8390 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 18:38:17.829147 8390 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 18:38:17.829197 8390 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 18:38:17.829224 8390 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 18:38:17.829241 8390 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 18:38:17.829254 8390 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 18:38:17.833976 8390 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.194.20:41559
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786041246094603-20232-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786041246094603-20232-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.19.194.62:46231
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.19.194.62:46231
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786041246094603-20232-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.19.194.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8390
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786041246094603-20232-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 17:59:13 UTC on ccycloud.quasar-wvsnzx.root.comops.site
build id 4571028
I20260806 18:38:17.834930 8390 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 18:38:17.836174 8390 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 18:38:17.844396 8395 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:38:17.844408 8396 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:38:17.844447 8399 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 18:38:17.844460 8390 server_base.cc:1034] running on GCE node
I20260806 18:38:17.845203 8390 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 18:38:17.846195 8390 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 18:38:17.847370 8390 hybrid_clock.cc:648] HybridClock initialized: now 1786041497847337 us; error 42 us; skew 500 ppm
I20260806 18:38:17.849171 8390 webserver.cc:460] Webserver started at http://127.19.194.62:37059/ using document root <none> and password file <none>
I20260806 18:38:17.849798 8390 fs_manager.cc:359] Metadata directory not provided
I20260806 18:38:17.849901 8390 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 18:38:17.850116 8390 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 18:38:17.852120 8390 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786041246094603-20232-0/minicluster-data/master-0/data/instance:
uuid: "5a2debea42ec454c8177fa1dc84c772f"
format_stamp: "Formatted at 2026-08-06 18:38:17 on dist-test-slave-1xrh"
server_key: "892ffef01f7c77d39dd0561cca16d7ea"
server_key_iv: "427a84927104e51293364b40f1dd5731"
server_key_version: "encryptionkey@0"
I20260806 18:38:17.852669 8390 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786041246094603-20232-0/minicluster-data/master-0/wal/instance:
uuid: "5a2debea42ec454c8177fa1dc84c772f"
format_stamp: "Formatted at 2026-08-06 18:38:17 on dist-test-slave-1xrh"
server_key: "892ffef01f7c77d39dd0561cca16d7ea"
server_key_iv: "427a84927104e51293364b40f1dd5731"
server_key_version: "encryptionkey@0"
I20260806 18:38:17.856215 8390 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.001s
I20260806 18:38:17.858532 8405 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 18:38:17.859683 8390 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260806 18:38:17.859818 8390 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786041246094603-20232-0/minicluster-data/master-0/data,/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786041246094603-20232-0/minicluster-data/master-0/wal
uuid: "5a2debea42ec454c8177fa1dc84c772f"
format_stamp: "Formatted at 2026-08-06 18:38:17 on dist-test-slave-1xrh"
server_key: "892ffef01f7c77d39dd0561cca16d7ea"
server_key_iv: "427a84927104e51293364b40f1dd5731"
server_key_version: "encryptionkey@0"
I20260806 18:38:17.859934 8390 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786041246094603-20232-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786041246094603-20232-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786041246094603-20232-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 18:38:17.869436 8390 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 18:38:17.870081 8390 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 18:38:17.870260 8390 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 18:38:17.877043 8390 rpc_server.cc:307] RPC server started. Bound to: 127.19.194.62:46231
I20260806 18:38:17.877079 8458 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.194.62:46231 every 8 connection(s)
I20260806 18:38:17.877892 8390 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786041246094603-20232-0/minicluster-data/master-0/data/info.pb
I20260806 18:38:17.880218 20232 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJW69tn/build/debug/bin/kudu as pid 8390
I20260806 18:38:17.880358 20232 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786041246094603-20232-0/minicluster-data/master-0/wal/instance
I20260806 18:38:17.880645 20232 external_mini_cluster.cc:1428] Setting key a305d4da35565df9b7fa7c36e03cfdc0
I20260806 18:38:17.880782 8459 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 18:38:17.887758 8459 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5a2debea42ec454c8177fa1dc84c772f: Bootstrap starting.
I20260806 18:38:17.890249 8459 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 5a2debea42ec454c8177fa1dc84c772f: Neither blocks nor log segments found. Creating new log.
I20260806 18:38:17.890627 8462 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:38:17.882036 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56692 (local address 127.19.194.62:46231)
0806 18:38:17.882693 (+ 657us) server_negotiation.cc:206] Beginning negotiation
0806 18:38:17.882712 (+ 19us) server_negotiation.cc:399] Waiting for connection header
0806 18:38:17.882752 (+ 40us) server_negotiation.cc:407] Connection header received
0806 18:38:17.883459 (+ 707us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:38:17.883481 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:38:17.883989 (+ 508us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:38:17.884336 (+ 347us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:38:17.884761 (+ 425us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:38:17.886251 (+ 1490us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:38:17.887203 (+ 952us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:38:17.887842 (+ 639us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:38:17.887990 (+ 148us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:38:17.888806 (+ 816us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:38:17.888814 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:38:17.888831 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 18:38:17.889286 (+ 455us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:38:17.889345 (+ 59us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 18:38:17.889369 (+ 24us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:38:17.889467 (+ 98us) server_negotiation.cc:1035] Waiting for connection context
0806 18:38:17.889547 (+ 80us) server_negotiation.cc:299] Negotiation successful
0806 18:38:17.889772 (+ 225us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":269,"thread_start_us":121,"threads_started":1}
I20260806 18:38:17.891062 8459 log.cc:824] T 00000000000000000000000000000000 P 5a2debea42ec454c8177fa1dc84c772f: Log is configured to *not* fsync() on all Append() calls
I20260806 18:38:17.892971 8459 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5a2debea42ec454c8177fa1dc84c772f: No bootstrap required, opened a new log
I20260806 18:38:17.896201 8459 raft_consensus.cc:348] T 00000000000000000000000000000000 P 5a2debea42ec454c8177fa1dc84c772f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5a2debea42ec454c8177fa1dc84c772f" member_type: VOTER last_known_addr { host: "127.19.194.62" port: 46231 } }
I20260806 18:38:17.896377 8459 raft_consensus.cc:374] T 00000000000000000000000000000000 P 5a2debea42ec454c8177fa1dc84c772f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 18:38:17.896407 8459 raft_consensus.cc:729] T 00000000000000000000000000000000 P 5a2debea42ec454c8177fa1dc84c772f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5a2debea42ec454c8177fa1dc84c772f, State: Initialized, Role: FOLLOWER
I20260806 18:38:17.896706 8459 consensus_queue.cc:260] T 00000000000000000000000000000000 P 5a2debea42ec454c8177fa1dc84c772f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5a2debea42ec454c8177fa1dc84c772f" member_type: VOTER last_known_addr { host: "127.19.194.62" port: 46231 } }
I20260806 18:38:17.896823 8459 raft_consensus.cc:388] T 00000000000000000000000000000000 P 5a2debea42ec454c8177fa1dc84c772f [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 18:38:17.896864 8459 raft_consensus.cc:482] T 00000000000000000000000000000000 P 5a2debea42ec454c8177fa1dc84c772f [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 18:38:17.896934 8459 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 5a2debea42ec454c8177fa1dc84c772f [term 0 FOLLOWER]: Advancing to term 1
I20260806 18:38:17.897655 8459 raft_consensus.cc:504] T 00000000000000000000000000000000 P 5a2debea42ec454c8177fa1dc84c772f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5a2debea42ec454c8177fa1dc84c772f" member_type: VOTER last_known_addr { host: "127.19.194.62" port: 46231 } }
I20260806 18:38:17.897930 8459 leader_election.cc:302] T 00000000000000000000000000000000 P 5a2debea42ec454c8177fa1dc84c772f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 5a2debea42ec454c8177fa1dc84c772f; no voters:
I20260806 18:38:17.898164 8459 leader_election.cc:288] T 00000000000000000000000000000000 P 5a2debea42ec454c8177fa1dc84c772f [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 18:38:17.898286 8464 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 5a2debea42ec454c8177fa1dc84c772f [term 1 FOLLOWER]: Leader election won for term 1
I20260806 18:38:17.898556 8464 raft_consensus.cc:686] T 00000000000000000000000000000000 P 5a2debea42ec454c8177fa1dc84c772f [term 1 LEADER]: Becoming Leader. State: Replica: 5a2debea42ec454c8177fa1dc84c772f, State: Running, Role: LEADER
I20260806 18:38:17.898864 8464 consensus_queue.cc:237] T 00000000000000000000000000000000 P 5a2debea42ec454c8177fa1dc84c772f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5a2debea42ec454c8177fa1dc84c772f" member_type: VOTER last_known_addr { host: "127.19.194.62" port: 46231 } }
I20260806 18:38:17.899307 8459 sys_catalog.cc:564] T 00000000000000000000000000000000 P 5a2debea42ec454c8177fa1dc84c772f [sys.catalog]: configured and running, proceeding with master startup.
I20260806 18:38:17.900563 8466 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5a2debea42ec454c8177fa1dc84c772f [sys.catalog]: SysCatalogTable state changed. Reason: New leader 5a2debea42ec454c8177fa1dc84c772f. Latest consensus state: current_term: 1 leader_uuid: "5a2debea42ec454c8177fa1dc84c772f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5a2debea42ec454c8177fa1dc84c772f" member_type: VOTER last_known_addr { host: "127.19.194.62" port: 46231 } } }
I20260806 18:38:17.900581 8465 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5a2debea42ec454c8177fa1dc84c772f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "5a2debea42ec454c8177fa1dc84c772f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5a2debea42ec454c8177fa1dc84c772f" member_type: VOTER last_known_addr { host: "127.19.194.62" port: 46231 } } }
I20260806 18:38:17.900686 8466 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5a2debea42ec454c8177fa1dc84c772f [sys.catalog]: This master's current role is: LEADER
I20260806 18:38:17.900699 8465 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5a2debea42ec454c8177fa1dc84c772f [sys.catalog]: This master's current role is: LEADER
I20260806 18:38:17.901043 8475 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 18:38:17.903684 8475 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 18:38:17.908649 8475 catalog_manager.cc:1347] Generated new cluster ID: 67d215311dbc4a78bce8cc64bb950473
I20260806 18:38:17.908704 8475 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 18:38:17.925415 8475 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 18:38:17.926239 8475 catalog_manager.cc:1504] Loading token signing keys...
I20260806 18:38:17.934114 8475 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 5a2debea42ec454c8177fa1dc84c772f: Generated new TSK 0
I20260806 18:38:17.934720 8475 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 18:38:17.941061 20232 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJW69tn/build/debug/bin/kudu
/tmp/dist-test-taskJW69tn/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786041246094603-20232-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786041246094603-20232-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786041246094603-20232-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786041246094603-20232-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.19.194.1:0
--local_ip_for_outbound_sockets=127.19.194.1
--webserver_interface=127.19.194.1
--webserver_port=0
--tserver_master_addrs=127.19.194.62:46231
--builtin_ntp_servers=127.19.194.20:41559
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260806 18:38:18.050351 8484 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 18:38:18.050577 8484 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 18:38:18.050637 8484 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 18:38:18.054378 8484 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 18:38:18.054419 8484 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 18:38:18.054466 8484 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.19.194.1
I20260806 18:38:18.059067 8484 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.194.20:41559
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786041246094603-20232-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786041246094603-20232-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.19.194.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786041246094603-20232-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.19.194.1
--webserver_port=0
--tserver_master_addrs=127.19.194.62:46231
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8484
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.19.194.1
--log_dir=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786041246094603-20232-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 17:59:13 UTC on ccycloud.quasar-wvsnzx.root.comops.site
build id 4571028
I20260806 18:38:18.059991 8484 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 18:38:18.061268 8484 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 18:38:18.070436 8490 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:38:18.070546 8492 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:38:18.070461 8489 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 18:38:18.070488 8484 server_base.cc:1034] running on GCE node
I20260806 18:38:18.070938 8484 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 18:38:18.071553 8484 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 18:38:18.072729 8484 hybrid_clock.cc:648] HybridClock initialized: now 1786041498072715 us; error 25 us; skew 500 ppm
I20260806 18:38:18.074316 8484 webserver.cc:460] Webserver started at http://127.19.194.1:41095/ using document root <none> and password file <none>
I20260806 18:38:18.074847 8484 fs_manager.cc:359] Metadata directory not provided
I20260806 18:38:18.074939 8484 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 18:38:18.075203 8484 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 18:38:18.077162 8484 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786041246094603-20232-0/minicluster-data/ts-0/data/instance:
uuid: "916b847b199249b297e62dfe70e8ff9d"
format_stamp: "Formatted at 2026-08-06 18:38:18 on dist-test-slave-1xrh"
server_key: "28668c30156ca440a8832855e0010da7"
server_key_iv: "bf89a0dbd545edfdea3b4bd0850856cd"
server_key_version: "encryptionkey@0"
I20260806 18:38:18.077672 8484 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786041246094603-20232-0/minicluster-data/ts-0/wal/instance:
uuid: "916b847b199249b297e62dfe70e8ff9d"
format_stamp: "Formatted at 2026-08-06 18:38:18 on dist-test-slave-1xrh"
server_key: "28668c30156ca440a8832855e0010da7"
server_key_iv: "bf89a0dbd545edfdea3b4bd0850856cd"
server_key_version: "encryptionkey@0"
I20260806 18:38:18.081126 8484 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.005s sys 0.000s
I20260806 18:38:18.083359 8498 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 18:38:18.084467 8484 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260806 18:38:18.084591 8484 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786041246094603-20232-0/minicluster-data/ts-0/data,/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786041246094603-20232-0/minicluster-data/ts-0/wal
uuid: "916b847b199249b297e62dfe70e8ff9d"
format_stamp: "Formatted at 2026-08-06 18:38:18 on dist-test-slave-1xrh"
server_key: "28668c30156ca440a8832855e0010da7"
server_key_iv: "bf89a0dbd545edfdea3b4bd0850856cd"
server_key_version: "encryptionkey@0"
I20260806 18:38:18.084697 8484 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786041246094603-20232-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786041246094603-20232-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786041246094603-20232-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 18:38:18.112294 8484 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 18:38:18.112942 8484 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 18:38:18.113129 8484 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 18:38:18.113683 8484 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 18:38:18.114558 8484 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 18:38:18.114624 8484 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 18:38:18.114698 8484 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 18:38:18.114734 8484 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 18:38:18.123334 8484 rpc_server.cc:307] RPC server started. Bound to: 127.19.194.1:32901
I20260806 18:38:18.123387 8619 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.194.1:32901 every 8 connection(s)
I20260806 18:38:18.124166 8484 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786041246094603-20232-0/minicluster-data/ts-0/data/info.pb
I20260806 18:38:18.127549 20232 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJW69tn/build/debug/bin/kudu as pid 8484
I20260806 18:38:18.127650 20232 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786041246094603-20232-0/minicluster-data/ts-0/wal/instance
I20260806 18:38:18.127946 20232 external_mini_cluster.cc:1428] Setting key 024ca61a3f468e6a82a9027fca2b278d
I20260806 18:38:18.129920 20232 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJW69tn/build/debug/bin/kudu
/tmp/dist-test-taskJW69tn/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786041246094603-20232-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786041246094603-20232-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786041246094603-20232-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786041246094603-20232-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.19.194.2:0
--local_ip_for_outbound_sockets=127.19.194.2
--webserver_interface=127.19.194.2
--webserver_port=0
--tserver_master_addrs=127.19.194.62:46231
--builtin_ntp_servers=127.19.194.20:41559
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260806 18:38:18.133690 8462 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:38:18.125801 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.194.1:52177 (local address 127.19.194.62:46231)
0806 18:38:18.125947 (+ 146us) server_negotiation.cc:206] Beginning negotiation
0806 18:38:18.125951 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 18:38:18.126909 (+ 958us) server_negotiation.cc:407] Connection header received
0806 18:38:18.128497 (+ 1588us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:38:18.128501 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:38:18.128557 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:38:18.128643 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:38:18.129724 (+ 1081us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:38:18.130345 (+ 621us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:38:18.131672 (+ 1327us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:38:18.131894 (+ 222us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:38:18.131977 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:38:18.133224 (+ 1247us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:38:18.133227 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:38:18.133229 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 18:38:18.133240 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:38:18.133260 (+ 20us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 18:38:18.133268 (+ 8us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:38:18.133324 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0806 18:38:18.133532 (+ 208us) server_negotiation.cc:299] Negotiation successful
0806 18:38:18.133583 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260806 18:38:18.134864 8622 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:38:18.126157 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.194.62:46231 (local address 127.19.194.1:52177)
0806 18:38:18.126758 (+ 601us) negotiation.cc:107] Waiting for socket to connect
0806 18:38:18.126799 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0806 18:38:18.128227 (+ 1428us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:38:18.128838 (+ 611us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:38:18.128846 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:38:18.128946 (+ 100us) client_negotiation.cc:189] Negotiated authn=SASL
0806 18:38:18.129579 (+ 633us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:38:18.129595 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:38:18.130471 (+ 876us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:38:18.130475 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:38:18.131545 (+ 1070us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:38:18.131560 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:38:18.132052 (+ 492us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:38:18.132058 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:38:18.132262 (+ 204us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:38:18.132940 (+ 678us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0806 18:38:18.132945 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0806 18:38:18.132997 (+ 52us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0806 18:38:18.133077 (+ 80us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 18:38:18.133350 (+ 273us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 18:38:18.133362 (+ 12us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 18:38:18.133371 (+ 9us) client_negotiation.cc:769] Sending connection context
0806 18:38:18.133526 (+ 155us) client_negotiation.cc:240] Negotiation successful
0806 18:38:18.133810 (+ 284us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":233,"thread_start_us":102,"threads_started":1}
I20260806 18:38:18.136037 8620 heartbeater.cc:347] Connected to a master server at 127.19.194.62:46231
I20260806 18:38:18.136286 8620 heartbeater.cc:464] Registering TS with master...
I20260806 18:38:18.136904 8620 heartbeater.cc:511] Master 127.19.194.62:46231 requested a full tablet report, sending...
I20260806 18:38:18.138320 8422 ts_manager.cc:194] Registered new tserver with Master: 916b847b199249b297e62dfe70e8ff9d (127.19.194.1:32901)
I20260806 18:38:18.139462 8422 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.19.194.1:52177
W20260806 18:38:18.239920 8623 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 18:38:18.240159 8623 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 18:38:18.240196 8623 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 18:38:18.244354 8623 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 18:38:18.244405 8623 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 18:38:18.244493 8623 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.19.194.2
I20260806 18:38:18.249222 8623 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.194.20:41559
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786041246094603-20232-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786041246094603-20232-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.19.194.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786041246094603-20232-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.19.194.2
--webserver_port=0
--tserver_master_addrs=127.19.194.62:46231
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8623
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.19.194.2
--log_dir=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786041246094603-20232-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 17:59:13 UTC on ccycloud.quasar-wvsnzx.root.comops.site
build id 4571028
I20260806 18:38:18.250276 8623 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 18:38:18.251677 8623 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 18:38:18.261060 8628 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:38:18.261070 8631 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:38:18.261092 8629 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 18:38:18.261104 8623 server_base.cc:1034] running on GCE node
I20260806 18:38:18.261686 8623 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 18:38:18.262283 8623 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 18:38:18.263458 8623 hybrid_clock.cc:648] HybridClock initialized: now 1786041498263448 us; error 30 us; skew 500 ppm
I20260806 18:38:18.265389 8623 webserver.cc:460] Webserver started at http://127.19.194.2:43319/ using document root <none> and password file <none>
I20260806 18:38:18.265945 8623 fs_manager.cc:359] Metadata directory not provided
I20260806 18:38:18.266036 8623 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 18:38:18.266252 8623 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 18:38:18.268304 8623 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786041246094603-20232-0/minicluster-data/ts-1/data/instance:
uuid: "c6ef26c2af3c4f86bd8359d8258b5ac9"
format_stamp: "Formatted at 2026-08-06 18:38:18 on dist-test-slave-1xrh"
server_key: "e861ab3b7484412d649bcfafa3eefb41"
server_key_iv: "a6e5a1df4f7466ce7ae390683c3102db"
server_key_version: "encryptionkey@0"
I20260806 18:38:18.268868 8623 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786041246094603-20232-0/minicluster-data/ts-1/wal/instance:
uuid: "c6ef26c2af3c4f86bd8359d8258b5ac9"
format_stamp: "Formatted at 2026-08-06 18:38:18 on dist-test-slave-1xrh"
server_key: "e861ab3b7484412d649bcfafa3eefb41"
server_key_iv: "a6e5a1df4f7466ce7ae390683c3102db"
server_key_version: "encryptionkey@0"
I20260806 18:38:18.272356 8623 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20260806 18:38:18.274683 8637 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 18:38:18.275801 8623 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260806 18:38:18.275944 8623 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786041246094603-20232-0/minicluster-data/ts-1/data,/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786041246094603-20232-0/minicluster-data/ts-1/wal
uuid: "c6ef26c2af3c4f86bd8359d8258b5ac9"
format_stamp: "Formatted at 2026-08-06 18:38:18 on dist-test-slave-1xrh"
server_key: "e861ab3b7484412d649bcfafa3eefb41"
server_key_iv: "a6e5a1df4f7466ce7ae390683c3102db"
server_key_version: "encryptionkey@0"
I20260806 18:38:18.276049 8623 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786041246094603-20232-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786041246094603-20232-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786041246094603-20232-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 18:38:18.296352 8623 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 18:38:18.297014 8623 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 18:38:18.297221 8623 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 18:38:18.297792 8623 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 18:38:18.298712 8623 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 18:38:18.298786 8623 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 18:38:18.298869 8623 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 18:38:18.298916 8623 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 18:38:18.307860 8623 rpc_server.cc:307] RPC server started. Bound to: 127.19.194.2:42903
I20260806 18:38:18.307896 8759 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.194.2:42903 every 8 connection(s)
I20260806 18:38:18.308706 8623 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786041246094603-20232-0/minicluster-data/ts-1/data/info.pb
I20260806 18:38:18.316885 20232 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJW69tn/build/debug/bin/kudu as pid 8623
I20260806 18:38:18.317029 20232 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786041246094603-20232-0/minicluster-data/ts-1/wal/instance
I20260806 18:38:18.317363 20232 external_mini_cluster.cc:1428] Setting key c24b81115eae6b074eb1e58589c4d16b
I20260806 18:38:18.317667 8462 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:38:18.310478 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.194.2:36035 (local address 127.19.194.62:46231)
0806 18:38:18.310602 (+ 124us) server_negotiation.cc:206] Beginning negotiation
0806 18:38:18.310605 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 18:38:18.311602 (+ 997us) server_negotiation.cc:407] Connection header received
0806 18:38:18.312987 (+ 1385us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:38:18.312990 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:38:18.313054 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:38:18.313146 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:38:18.314002 (+ 856us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:38:18.314513 (+ 511us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:38:18.315683 (+ 1170us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:38:18.315905 (+ 222us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:38:18.316015 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:38:18.317183 (+ 1168us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:38:18.317186 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:38:18.317188 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 18:38:18.317201 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:38:18.317221 (+ 20us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 18:38:18.317229 (+ 8us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:38:18.317293 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0806 18:38:18.317489 (+ 196us) server_negotiation.cc:299] Negotiation successful
0806 18:38:18.317554 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260806 18:38:18.318805 8762 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:38:18.310786 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.194.62:46231 (local address 127.19.194.2:36035)
0806 18:38:18.311466 (+ 680us) negotiation.cc:107] Waiting for socket to connect
0806 18:38:18.311506 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0806 18:38:18.312770 (+ 1264us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:38:18.313311 (+ 541us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:38:18.313317 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:38:18.313393 (+ 76us) client_negotiation.cc:189] Negotiated authn=SASL
0806 18:38:18.313859 (+ 466us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:38:18.313872 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:38:18.314646 (+ 774us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:38:18.314649 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:38:18.315555 (+ 906us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:38:18.315567 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:38:18.316056 (+ 489us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:38:18.316072 (+ 16us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:38:18.316221 (+ 149us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:38:18.316823 (+ 602us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0806 18:38:18.316825 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0806 18:38:18.316921 (+ 96us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0806 18:38:18.317008 (+ 87us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 18:38:18.317319 (+ 311us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 18:38:18.317329 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 18:38:18.317335 (+ 6us) client_negotiation.cc:769] Sending connection context
0806 18:38:18.317472 (+ 137us) client_negotiation.cc:240] Negotiation successful
0806 18:38:18.317737 (+ 265us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":245,"thread_start_us":116,"threads_started":1}
I20260806 18:38:18.319502 20232 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJW69tn/build/debug/bin/kudu
/tmp/dist-test-taskJW69tn/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786041246094603-20232-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786041246094603-20232-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786041246094603-20232-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786041246094603-20232-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.19.194.3:0
--local_ip_for_outbound_sockets=127.19.194.3
--webserver_interface=127.19.194.3
--webserver_port=0
--tserver_master_addrs=127.19.194.62:46231
--builtin_ntp_servers=127.19.194.20:41559
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260806 18:38:18.319921 8760 heartbeater.cc:347] Connected to a master server at 127.19.194.62:46231
I20260806 18:38:18.320152 8760 heartbeater.cc:464] Registering TS with master...
I20260806 18:38:18.320627 8760 heartbeater.cc:511] Master 127.19.194.62:46231 requested a full tablet report, sending...
I20260806 18:38:18.321699 8422 ts_manager.cc:194] Registered new tserver with Master: c6ef26c2af3c4f86bd8359d8258b5ac9 (127.19.194.2:42903)
I20260806 18:38:18.322288 8422 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.19.194.2:36035
W20260806 18:38:18.432770 8763 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 18:38:18.433005 8763 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 18:38:18.433041 8763 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 18:38:18.436995 8763 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 18:38:18.437037 8763 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 18:38:18.437088 8763 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.19.194.3
I20260806 18:38:18.441711 8763 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.194.20:41559
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786041246094603-20232-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786041246094603-20232-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.19.194.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786041246094603-20232-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.19.194.3
--webserver_port=0
--tserver_master_addrs=127.19.194.62:46231
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8763
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.19.194.3
--log_dir=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786041246094603-20232-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 17:59:13 UTC on ccycloud.quasar-wvsnzx.root.comops.site
build id 4571028
I20260806 18:38:18.442615 8763 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 18:38:18.443974 8763 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 18:38:18.453178 8774 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:38:18.453179 8771 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:38:18.453197 8772 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 18:38:18.453387 8763 server_base.cc:1034] running on GCE node
I20260806 18:38:18.453881 8763 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 18:38:18.454483 8763 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 18:38:18.455636 8763 hybrid_clock.cc:648] HybridClock initialized: now 1786041498455608 us; error 41 us; skew 500 ppm
I20260806 18:38:18.457456 8763 webserver.cc:460] Webserver started at http://127.19.194.3:45403/ using document root <none> and password file <none>
I20260806 18:38:18.457996 8763 fs_manager.cc:359] Metadata directory not provided
I20260806 18:38:18.458053 8763 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 18:38:18.458279 8763 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 18:38:18.460289 8763 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786041246094603-20232-0/minicluster-data/ts-2/data/instance:
uuid: "a645dd38cfa74cf586a45ea98a504b1f"
format_stamp: "Formatted at 2026-08-06 18:38:18 on dist-test-slave-1xrh"
server_key: "d27e18e8cab76f00e61850651715fb63"
server_key_iv: "a4858e980566822ff3fb16ad2508e88f"
server_key_version: "encryptionkey@0"
I20260806 18:38:18.460820 8763 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786041246094603-20232-0/minicluster-data/ts-2/wal/instance:
uuid: "a645dd38cfa74cf586a45ea98a504b1f"
format_stamp: "Formatted at 2026-08-06 18:38:18 on dist-test-slave-1xrh"
server_key: "d27e18e8cab76f00e61850651715fb63"
server_key_iv: "a4858e980566822ff3fb16ad2508e88f"
server_key_version: "encryptionkey@0"
I20260806 18:38:18.464263 8763 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20260806 18:38:18.466447 8780 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 18:38:18.467689 8763 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260806 18:38:18.467808 8763 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786041246094603-20232-0/minicluster-data/ts-2/data,/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786041246094603-20232-0/minicluster-data/ts-2/wal
uuid: "a645dd38cfa74cf586a45ea98a504b1f"
format_stamp: "Formatted at 2026-08-06 18:38:18 on dist-test-slave-1xrh"
server_key: "d27e18e8cab76f00e61850651715fb63"
server_key_iv: "a4858e980566822ff3fb16ad2508e88f"
server_key_version: "encryptionkey@0"
I20260806 18:38:18.467907 8763 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786041246094603-20232-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786041246094603-20232-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786041246094603-20232-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 18:38:18.477698 8763 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 18:38:18.478215 8763 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 18:38:18.478371 8763 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 18:38:18.478869 8763 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 18:38:18.479780 8763 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 18:38:18.479856 8763 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 18:38:18.479918 8763 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 18:38:18.479961 8763 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 18:38:18.488502 8763 rpc_server.cc:307] RPC server started. Bound to: 127.19.194.3:33405
I20260806 18:38:18.488534 8906 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.194.3:33405 every 8 connection(s)
I20260806 18:38:18.489367 8763 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786041246094603-20232-0/minicluster-data/ts-2/data/info.pb
I20260806 18:38:18.496621 20232 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJW69tn/build/debug/bin/kudu as pid 8763
I20260806 18:38:18.496732 20232 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786041246094603-20232-0/minicluster-data/ts-2/wal/instance
I20260806 18:38:18.497092 20232 external_mini_cluster.cc:1428] Setting key f85432c2e09d452acc327a4f3d3fd149
I20260806 18:38:18.497743 8462 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:38:18.491067 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.194.3:43487 (local address 127.19.194.62:46231)
0806 18:38:18.491237 (+ 170us) server_negotiation.cc:206] Beginning negotiation
0806 18:38:18.491241 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 18:38:18.492193 (+ 952us) server_negotiation.cc:407] Connection header received
0806 18:38:18.493455 (+ 1262us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:38:18.493457 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:38:18.493505 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:38:18.493575 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:38:18.494344 (+ 769us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:38:18.494813 (+ 469us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:38:18.495823 (+ 1010us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:38:18.496036 (+ 213us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:38:18.496110 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:38:18.497297 (+ 1187us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:38:18.497301 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:38:18.497303 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 18:38:18.497314 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:38:18.497330 (+ 16us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 18:38:18.497338 (+ 8us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:38:18.497409 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0806 18:38:18.497563 (+ 154us) server_negotiation.cc:299] Negotiation successful
0806 18:38:18.497625 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260806 18:38:18.498601 8909 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:38:18.491417 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.194.62:46231 (local address 127.19.194.3:43487)
0806 18:38:18.492044 (+ 627us) negotiation.cc:107] Waiting for socket to connect
0806 18:38:18.492082 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0806 18:38:18.493283 (+ 1201us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:38:18.493724 (+ 441us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:38:18.493745 (+ 21us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:38:18.493818 (+ 73us) client_negotiation.cc:189] Negotiated authn=SASL
0806 18:38:18.494233 (+ 415us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:38:18.494244 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:38:18.494921 (+ 677us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:38:18.494923 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:38:18.495718 (+ 795us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:38:18.495727 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:38:18.496158 (+ 431us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:38:18.496161 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:38:18.496306 (+ 145us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:38:18.497001 (+ 695us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0806 18:38:18.497003 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0806 18:38:18.497053 (+ 50us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0806 18:38:18.497138 (+ 85us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 18:38:18.497420 (+ 282us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 18:38:18.497430 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 18:38:18.497444 (+ 14us) client_negotiation.cc:769] Sending connection context
0806 18:38:18.497538 (+ 94us) client_negotiation.cc:240] Negotiation successful
0806 18:38:18.497790 (+ 252us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":246,"thread_start_us":111,"threads_started":1}
I20260806 18:38:18.499774 8907 heartbeater.cc:347] Connected to a master server at 127.19.194.62:46231
I20260806 18:38:18.499986 8907 heartbeater.cc:464] Registering TS with master...
I20260806 18:38:18.500485 8907 heartbeater.cc:511] Master 127.19.194.62:46231 requested a full tablet report, sending...
I20260806 18:38:18.501433 8422 ts_manager.cc:194] Registered new tserver with Master: a645dd38cfa74cf586a45ea98a504b1f (127.19.194.3:33405)
I20260806 18:38:18.501971 8422 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.19.194.3:43487
I20260806 18:38:18.510877 20232 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260806 18:38:18.515936 8462 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:38:18.512768 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56694 (local address 127.19.194.62:46231)
0806 18:38:18.512897 (+ 129us) server_negotiation.cc:206] Beginning negotiation
0806 18:38:18.512903 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 18:38:18.513033 (+ 130us) server_negotiation.cc:407] Connection header received
0806 18:38:18.513119 (+ 86us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:38:18.513121 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:38:18.513156 (+ 35us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:38:18.513215 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:38:18.513453 (+ 238us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:38:18.513896 (+ 443us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:38:18.514618 (+ 722us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:38:18.514806 (+ 188us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:38:18.514874 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:38:18.515574 (+ 700us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:38:18.515577 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:38:18.515579 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 18:38:18.515589 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:38:18.515605 (+ 16us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 18:38:18.515612 (+ 7us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:38:18.515668 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0806 18:38:18.515739 (+ 71us) server_negotiation.cc:299] Negotiation successful
0806 18:38:18.515781 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260806 18:38:18.518307 8422 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:56694:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260806 18:38:18.521257 8422 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 18:38:18.535312 8924 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:38:18.529569 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60238 (local address 127.19.194.1:32901)
0806 18:38:18.529948 (+ 379us) server_negotiation.cc:206] Beginning negotiation
0806 18:38:18.529963 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0806 18:38:18.529988 (+ 25us) server_negotiation.cc:407] Connection header received
0806 18:38:18.530066 (+ 78us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:38:18.530071 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:38:18.530264 (+ 193us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:38:18.530773 (+ 509us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 18:38:18.530933 (+ 160us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:38:18.532065 (+ 1132us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:38:18.533706 (+ 1641us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:38:18.534508 (+ 802us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:38:18.534624 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:38:18.534805 (+ 181us) server_negotiation.cc:1035] Waiting for connection context
0806 18:38:18.535008 (+ 203us) server_negotiation.cc:299] Negotiation successful
0806 18:38:18.535114 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":261,"thread_start_us":137,"threads_started":1}
I20260806 18:38:18.535310 8922 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:38:18.529000 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.194.1:32901 (local address 127.0.0.1:60238)
0806 18:38:18.529270 (+ 270us) negotiation.cc:107] Waiting for socket to connect
0806 18:38:18.529284 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0806 18:38:18.529390 (+ 106us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:38:18.530452 (+ 1062us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:38:18.530458 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:38:18.530479 (+ 21us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 18:38:18.530613 (+ 134us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:38:18.530622 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:38:18.532193 (+ 1571us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:38:18.532200 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:38:18.533585 (+ 1385us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:38:18.533595 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:38:18.534636 (+ 1041us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:38:18.534639 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:38:18.534803 (+ 164us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:38:18.534862 (+ 59us) client_negotiation.cc:769] Sending connection context
0806 18:38:18.534968 (+ 106us) client_negotiation.cc:240] Negotiation successful
0806 18:38:18.535151 (+ 183us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":212,"mutex_wait_us":323,"thread_start_us":111,"threads_started":1}
I20260806 18:38:18.535310 8921 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:38:18.528799 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.194.2:42903 (local address 127.0.0.1:38482)
0806 18:38:18.529110 (+ 311us) negotiation.cc:107] Waiting for socket to connect
0806 18:38:18.529160 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0806 18:38:18.529282 (+ 122us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:38:18.530563 (+ 1281us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:38:18.530566 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:38:18.530580 (+ 14us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 18:38:18.530722 (+ 142us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:38:18.530728 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:38:18.532229 (+ 1501us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:38:18.532232 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:38:18.533676 (+ 1444us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:38:18.533688 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:38:18.534636 (+ 948us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:38:18.534639 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:38:18.534800 (+ 161us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:38:18.534857 (+ 57us) client_negotiation.cc:769] Sending connection context
0806 18:38:18.534978 (+ 121us) client_negotiation.cc:240] Negotiation successful
0806 18:38:18.535143 (+ 165us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":137,"mutex_wait_us":190,"thread_start_us":61,"threads_started":1}
I20260806 18:38:18.535809 8923 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:38:18.529453 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38482 (local address 127.19.194.2:42903)
0806 18:38:18.529953 (+ 500us) server_negotiation.cc:206] Beginning negotiation
0806 18:38:18.529965 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0806 18:38:18.529988 (+ 23us) server_negotiation.cc:407] Connection header received
0806 18:38:18.530066 (+ 78us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:38:18.530071 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:38:18.530264 (+ 193us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:38:18.530418 (+ 154us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 18:38:18.530849 (+ 431us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:38:18.532081 (+ 1232us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:38:18.533787 (+ 1706us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:38:18.534508 (+ 721us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:38:18.534624 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:38:18.534805 (+ 181us) server_negotiation.cc:1035] Waiting for connection context
0806 18:38:18.535542 (+ 737us) server_negotiation.cc:299] Negotiation successful
0806 18:38:18.535644 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":394,"thread_start_us":146,"threads_started":1}
I20260806 18:38:18.536316 8926 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:38:18.529639 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.194.3:33405 (local address 127.0.0.1:51726)
0806 18:38:18.530216 (+ 577us) negotiation.cc:107] Waiting for socket to connect
0806 18:38:18.530231 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0806 18:38:18.530316 (+ 85us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:38:18.531160 (+ 844us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:38:18.531163 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:38:18.531182 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 18:38:18.531267 (+ 85us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:38:18.531275 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:38:18.532239 (+ 964us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:38:18.532242 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:38:18.533676 (+ 1434us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:38:18.533688 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:38:18.535930 (+ 2242us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:38:18.535934 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:38:18.536031 (+ 97us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:38:18.536056 (+ 25us) client_negotiation.cc:769] Sending connection context
0806 18:38:18.536113 (+ 57us) client_negotiation.cc:240] Negotiation successful
0806 18:38:18.536184 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":516,"mutex_wait_us":111,"thread_start_us":90,"threads_started":1}
I20260806 18:38:18.537066 8927 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:38:18.530038 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51726 (local address 127.19.194.3:33405)
0806 18:38:18.530747 (+ 709us) server_negotiation.cc:206] Beginning negotiation
0806 18:38:18.530761 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0806 18:38:18.530785 (+ 24us) server_negotiation.cc:407] Connection header received
0806 18:38:18.530855 (+ 70us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:38:18.530865 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:38:18.531008 (+ 143us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:38:18.531186 (+ 178us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 18:38:18.531386 (+ 200us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:38:18.532084 (+ 698us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:38:18.534866 (+ 2782us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:38:18.535779 (+ 913us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:38:18.536606 (+ 827us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:38:18.536788 (+ 182us) server_negotiation.cc:1035] Waiting for connection context
0806 18:38:18.536838 (+ 50us) server_negotiation.cc:299] Negotiation successful
0806 18:38:18.536920 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":608,"thread_start_us":97,"threads_started":1}
I20260806 18:38:18.538558 8689 tablet_service.cc:1467] Processing CreateTablet for tablet a0976eb3149941de9b2d864dc8cd0fcc (DEFAULT_TABLE table=test-table [id=29c052cb96f449ca82f731e87cedeb75]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 18:38:18.538622 8549 tablet_service.cc:1467] Processing CreateTablet for tablet a0976eb3149941de9b2d864dc8cd0fcc (DEFAULT_TABLE table=test-table [id=29c052cb96f449ca82f731e87cedeb75]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 18:38:18.539597 8689 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a0976eb3149941de9b2d864dc8cd0fcc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 18:38:18.539674 8549 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a0976eb3149941de9b2d864dc8cd0fcc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 18:38:18.539503 8834 tablet_service.cc:1467] Processing CreateTablet for tablet a0976eb3149941de9b2d864dc8cd0fcc (DEFAULT_TABLE table=test-table [id=29c052cb96f449ca82f731e87cedeb75]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 18:38:18.540340 8834 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a0976eb3149941de9b2d864dc8cd0fcc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 18:38:18.545642 8928 tablet_bootstrap.cc:492] T a0976eb3149941de9b2d864dc8cd0fcc P c6ef26c2af3c4f86bd8359d8258b5ac9: Bootstrap starting.
I20260806 18:38:18.547055 8930 tablet_bootstrap.cc:492] T a0976eb3149941de9b2d864dc8cd0fcc P 916b847b199249b297e62dfe70e8ff9d: Bootstrap starting.
I20260806 18:38:18.547410 8929 tablet_bootstrap.cc:492] T a0976eb3149941de9b2d864dc8cd0fcc P a645dd38cfa74cf586a45ea98a504b1f: Bootstrap starting.
I20260806 18:38:18.547897 8928 tablet_bootstrap.cc:654] T a0976eb3149941de9b2d864dc8cd0fcc P c6ef26c2af3c4f86bd8359d8258b5ac9: Neither blocks nor log segments found. Creating new log.
I20260806 18:38:18.548689 8928 log.cc:824] T a0976eb3149941de9b2d864dc8cd0fcc P c6ef26c2af3c4f86bd8359d8258b5ac9: Log is configured to *not* fsync() on all Append() calls
I20260806 18:38:18.549005 8930 tablet_bootstrap.cc:654] T a0976eb3149941de9b2d864dc8cd0fcc P 916b847b199249b297e62dfe70e8ff9d: Neither blocks nor log segments found. Creating new log.
I20260806 18:38:18.549360 8929 tablet_bootstrap.cc:654] T a0976eb3149941de9b2d864dc8cd0fcc P a645dd38cfa74cf586a45ea98a504b1f: Neither blocks nor log segments found. Creating new log.
I20260806 18:38:18.549830 8930 log.cc:824] T a0976eb3149941de9b2d864dc8cd0fcc P 916b847b199249b297e62dfe70e8ff9d: Log is configured to *not* fsync() on all Append() calls
I20260806 18:38:18.550098 8929 log.cc:824] T a0976eb3149941de9b2d864dc8cd0fcc P a645dd38cfa74cf586a45ea98a504b1f: Log is configured to *not* fsync() on all Append() calls
I20260806 18:38:18.550413 8928 tablet_bootstrap.cc:492] T a0976eb3149941de9b2d864dc8cd0fcc P c6ef26c2af3c4f86bd8359d8258b5ac9: No bootstrap required, opened a new log
I20260806 18:38:18.550633 8928 ts_tablet_manager.cc:1397] T a0976eb3149941de9b2d864dc8cd0fcc P c6ef26c2af3c4f86bd8359d8258b5ac9: Time spent bootstrapping tablet: real 0.005s user 0.005s sys 0.000s
I20260806 18:38:18.551503 8930 tablet_bootstrap.cc:492] T a0976eb3149941de9b2d864dc8cd0fcc P 916b847b199249b297e62dfe70e8ff9d: No bootstrap required, opened a new log
I20260806 18:38:18.551681 8930 ts_tablet_manager.cc:1397] T a0976eb3149941de9b2d864dc8cd0fcc P 916b847b199249b297e62dfe70e8ff9d: Time spent bootstrapping tablet: real 0.005s user 0.001s sys 0.003s
I20260806 18:38:18.551977 8929 tablet_bootstrap.cc:492] T a0976eb3149941de9b2d864dc8cd0fcc P a645dd38cfa74cf586a45ea98a504b1f: No bootstrap required, opened a new log
I20260806 18:38:18.552152 8929 ts_tablet_manager.cc:1397] T a0976eb3149941de9b2d864dc8cd0fcc P a645dd38cfa74cf586a45ea98a504b1f: Time spent bootstrapping tablet: real 0.005s user 0.003s sys 0.000s
I20260806 18:38:18.554692 8930 raft_consensus.cc:348] T a0976eb3149941de9b2d864dc8cd0fcc P 916b847b199249b297e62dfe70e8ff9d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c6ef26c2af3c4f86bd8359d8258b5ac9" member_type: VOTER last_known_addr { host: "127.19.194.2" port: 42903 } } peers { permanent_uuid: "916b847b199249b297e62dfe70e8ff9d" member_type: VOTER last_known_addr { host: "127.19.194.1" port: 32901 } } peers { permanent_uuid: "a645dd38cfa74cf586a45ea98a504b1f" member_type: VOTER last_known_addr { host: "127.19.194.3" port: 33405 } }
I20260806 18:38:18.554904 8930 raft_consensus.cc:374] T a0976eb3149941de9b2d864dc8cd0fcc P 916b847b199249b297e62dfe70e8ff9d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 18:38:18.554973 8930 raft_consensus.cc:729] T a0976eb3149941de9b2d864dc8cd0fcc P 916b847b199249b297e62dfe70e8ff9d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 916b847b199249b297e62dfe70e8ff9d, State: Initialized, Role: FOLLOWER
I20260806 18:38:18.554901 8928 raft_consensus.cc:348] T a0976eb3149941de9b2d864dc8cd0fcc P c6ef26c2af3c4f86bd8359d8258b5ac9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c6ef26c2af3c4f86bd8359d8258b5ac9" member_type: VOTER last_known_addr { host: "127.19.194.2" port: 42903 } } peers { permanent_uuid: "916b847b199249b297e62dfe70e8ff9d" member_type: VOTER last_known_addr { host: "127.19.194.1" port: 32901 } } peers { permanent_uuid: "a645dd38cfa74cf586a45ea98a504b1f" member_type: VOTER last_known_addr { host: "127.19.194.3" port: 33405 } }
I20260806 18:38:18.555181 8928 raft_consensus.cc:374] T a0976eb3149941de9b2d864dc8cd0fcc P c6ef26c2af3c4f86bd8359d8258b5ac9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 18:38:18.555240 8928 raft_consensus.cc:729] T a0976eb3149941de9b2d864dc8cd0fcc P c6ef26c2af3c4f86bd8359d8258b5ac9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c6ef26c2af3c4f86bd8359d8258b5ac9, State: Initialized, Role: FOLLOWER
I20260806 18:38:18.555478 8930 consensus_queue.cc:260] T a0976eb3149941de9b2d864dc8cd0fcc P 916b847b199249b297e62dfe70e8ff9d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c6ef26c2af3c4f86bd8359d8258b5ac9" member_type: VOTER last_known_addr { host: "127.19.194.2" port: 42903 } } peers { permanent_uuid: "916b847b199249b297e62dfe70e8ff9d" member_type: VOTER last_known_addr { host: "127.19.194.1" port: 32901 } } peers { permanent_uuid: "a645dd38cfa74cf586a45ea98a504b1f" member_type: VOTER last_known_addr { host: "127.19.194.3" port: 33405 } }
I20260806 18:38:18.555632 8928 consensus_queue.cc:260] T a0976eb3149941de9b2d864dc8cd0fcc P c6ef26c2af3c4f86bd8359d8258b5ac9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c6ef26c2af3c4f86bd8359d8258b5ac9" member_type: VOTER last_known_addr { host: "127.19.194.2" port: 42903 } } peers { permanent_uuid: "916b847b199249b297e62dfe70e8ff9d" member_type: VOTER last_known_addr { host: "127.19.194.1" port: 32901 } } peers { permanent_uuid: "a645dd38cfa74cf586a45ea98a504b1f" member_type: VOTER last_known_addr { host: "127.19.194.3" port: 33405 } }
I20260806 18:38:18.556341 8620 heartbeater.cc:503] Master 127.19.194.62:46231 was elected leader, sending a full tablet report...
I20260806 18:38:18.556465 8930 ts_tablet_manager.cc:1428] T a0976eb3149941de9b2d864dc8cd0fcc P 916b847b199249b297e62dfe70e8ff9d: Time spent starting tablet: real 0.005s user 0.005s sys 0.000s
I20260806 18:38:18.556555 8928 ts_tablet_manager.cc:1428] T a0976eb3149941de9b2d864dc8cd0fcc P c6ef26c2af3c4f86bd8359d8258b5ac9: Time spent starting tablet: real 0.006s user 0.005s sys 0.000s
I20260806 18:38:18.556877 8760 heartbeater.cc:503] Master 127.19.194.62:46231 was elected leader, sending a full tablet report...
I20260806 18:38:18.556985 8929 raft_consensus.cc:348] T a0976eb3149941de9b2d864dc8cd0fcc P a645dd38cfa74cf586a45ea98a504b1f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c6ef26c2af3c4f86bd8359d8258b5ac9" member_type: VOTER last_known_addr { host: "127.19.194.2" port: 42903 } } peers { permanent_uuid: "916b847b199249b297e62dfe70e8ff9d" member_type: VOTER last_known_addr { host: "127.19.194.1" port: 32901 } } peers { permanent_uuid: "a645dd38cfa74cf586a45ea98a504b1f" member_type: VOTER last_known_addr { host: "127.19.194.3" port: 33405 } }
I20260806 18:38:18.557317 8929 raft_consensus.cc:374] T a0976eb3149941de9b2d864dc8cd0fcc P a645dd38cfa74cf586a45ea98a504b1f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 18:38:18.557410 8929 raft_consensus.cc:729] T a0976eb3149941de9b2d864dc8cd0fcc P a645dd38cfa74cf586a45ea98a504b1f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a645dd38cfa74cf586a45ea98a504b1f, State: Initialized, Role: FOLLOWER
I20260806 18:38:18.557832 8929 consensus_queue.cc:260] T a0976eb3149941de9b2d864dc8cd0fcc P a645dd38cfa74cf586a45ea98a504b1f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c6ef26c2af3c4f86bd8359d8258b5ac9" member_type: VOTER last_known_addr { host: "127.19.194.2" port: 42903 } } peers { permanent_uuid: "916b847b199249b297e62dfe70e8ff9d" member_type: VOTER last_known_addr { host: "127.19.194.1" port: 32901 } } peers { permanent_uuid: "a645dd38cfa74cf586a45ea98a504b1f" member_type: VOTER last_known_addr { host: "127.19.194.3" port: 33405 } }
I20260806 18:38:18.558462 8907 heartbeater.cc:503] Master 127.19.194.62:46231 was elected leader, sending a full tablet report...
I20260806 18:38:18.558773 8929 ts_tablet_manager.cc:1428] T a0976eb3149941de9b2d864dc8cd0fcc P a645dd38cfa74cf586a45ea98a504b1f: Time spent starting tablet: real 0.007s user 0.006s sys 0.000s
W20260806 18:38:18.559542 8761 tablet.cc:2369] T a0976eb3149941de9b2d864dc8cd0fcc P c6ef26c2af3c4f86bd8359d8258b5ac9: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260806 18:38:18.625244 8621 tablet.cc:2369] T a0976eb3149941de9b2d864dc8cd0fcc P 916b847b199249b297e62dfe70e8ff9d: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 18:38:18.668699 8938 raft_consensus.cc:482] T a0976eb3149941de9b2d864dc8cd0fcc P a645dd38cfa74cf586a45ea98a504b1f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 18:38:18.668944 8938 raft_consensus.cc:504] T a0976eb3149941de9b2d864dc8cd0fcc P a645dd38cfa74cf586a45ea98a504b1f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c6ef26c2af3c4f86bd8359d8258b5ac9" member_type: VOTER last_known_addr { host: "127.19.194.2" port: 42903 } } peers { permanent_uuid: "916b847b199249b297e62dfe70e8ff9d" member_type: VOTER last_known_addr { host: "127.19.194.1" port: 32901 } } peers { permanent_uuid: "a645dd38cfa74cf586a45ea98a504b1f" member_type: VOTER last_known_addr { host: "127.19.194.3" port: 33405 } }
I20260806 18:38:18.670109 8938 leader_election.cc:288] T a0976eb3149941de9b2d864dc8cd0fcc P a645dd38cfa74cf586a45ea98a504b1f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c6ef26c2af3c4f86bd8359d8258b5ac9 (127.19.194.2:42903), 916b847b199249b297e62dfe70e8ff9d (127.19.194.1:32901)
I20260806 18:38:18.673784 8909 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:38:18.670378 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.194.1:32901 (local address 127.19.194.3:54045)
0806 18:38:18.670519 (+ 141us) negotiation.cc:107] Waiting for socket to connect
0806 18:38:18.670535 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0806 18:38:18.670649 (+ 114us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:38:18.670935 (+ 286us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:38:18.670938 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:38:18.670957 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 18:38:18.671078 (+ 121us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:38:18.671084 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:38:18.671926 (+ 842us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:38:18.671931 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:38:18.672756 (+ 825us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:38:18.672766 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:38:18.673383 (+ 617us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:38:18.673387 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:38:18.673480 (+ 93us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:38:18.673505 (+ 25us) client_negotiation.cc:769] Sending connection context
0806 18:38:18.673586 (+ 81us) client_negotiation.cc:240] Negotiation successful
0806 18:38:18.673654 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":50}
I20260806 18:38:18.673784 8924 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:38:18.670460 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.194.3:54045 (local address 127.19.194.1:32901)
0806 18:38:18.670597 (+ 137us) server_negotiation.cc:206] Beginning negotiation
0806 18:38:18.670602 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 18:38:18.670619 (+ 17us) server_negotiation.cc:407] Connection header received
0806 18:38:18.670763 (+ 144us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:38:18.670765 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:38:18.670821 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:38:18.670919 (+ 98us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 18:38:18.671232 (+ 313us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:38:18.671810 (+ 578us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:38:18.672881 (+ 1071us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:38:18.673266 (+ 385us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:38:18.673389 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:38:18.673439 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0806 18:38:18.673604 (+ 165us) server_negotiation.cc:299] Negotiation successful
0806 18:38:18.673669 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260806 18:38:18.674312 8939 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:38:18.670378 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.194.2:42903 (local address 127.19.194.3:50059)
0806 18:38:18.671098 (+ 720us) negotiation.cc:107] Waiting for socket to connect
0806 18:38:18.671109 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0806 18:38:18.671244 (+ 135us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:38:18.671519 (+ 275us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:38:18.671521 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:38:18.671537 (+ 16us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 18:38:18.671701 (+ 164us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:38:18.671707 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:38:18.672498 (+ 791us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:38:18.672502 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:38:18.673242 (+ 740us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:38:18.673250 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:38:18.673880 (+ 630us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:38:18.673883 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:38:18.673983 (+ 100us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:38:18.674008 (+ 25us) client_negotiation.cc:769] Sending connection context
0806 18:38:18.674109 (+ 101us) client_negotiation.cc:240] Negotiation successful
0806 18:38:18.674171 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":656,"mutex_wait_us":357,"thread_start_us":152,"threads_started":1}
I20260806 18:38:18.674300 8923 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:38:18.670585 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.194.3:50059 (local address 127.19.194.2:42903)
0806 18:38:18.670749 (+ 164us) server_negotiation.cc:206] Beginning negotiation
0806 18:38:18.670755 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 18:38:18.671209 (+ 454us) server_negotiation.cc:407] Connection header received
0806 18:38:18.671374 (+ 165us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:38:18.671377 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:38:18.671426 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:38:18.671507 (+ 81us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 18:38:18.671810 (+ 303us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:38:18.672388 (+ 578us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:38:18.673363 (+ 975us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:38:18.673741 (+ 378us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:38:18.673871 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:38:18.673926 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0806 18:38:18.674130 (+ 204us) server_negotiation.cc:299] Negotiation successful
0806 18:38:18.674199 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260806 18:38:18.674480 8570 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a0976eb3149941de9b2d864dc8cd0fcc" candidate_uuid: "a645dd38cfa74cf586a45ea98a504b1f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "916b847b199249b297e62dfe70e8ff9d" is_pre_election: true
I20260806 18:38:18.674784 8570 raft_consensus.cc:2454] T a0976eb3149941de9b2d864dc8cd0fcc P 916b847b199249b297e62dfe70e8ff9d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a645dd38cfa74cf586a45ea98a504b1f in term 0.
I20260806 18:38:18.674888 8711 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a0976eb3149941de9b2d864dc8cd0fcc" candidate_uuid: "a645dd38cfa74cf586a45ea98a504b1f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c6ef26c2af3c4f86bd8359d8258b5ac9" is_pre_election: true
I20260806 18:38:18.675201 8711 raft_consensus.cc:2454] T a0976eb3149941de9b2d864dc8cd0fcc P c6ef26c2af3c4f86bd8359d8258b5ac9 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a645dd38cfa74cf586a45ea98a504b1f in term 0.
I20260806 18:38:18.675351 8785 leader_election.cc:302] T a0976eb3149941de9b2d864dc8cd0fcc P a645dd38cfa74cf586a45ea98a504b1f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 916b847b199249b297e62dfe70e8ff9d, a645dd38cfa74cf586a45ea98a504b1f; no voters:
I20260806 18:38:18.675653 8938 raft_consensus.cc:2781] T a0976eb3149941de9b2d864dc8cd0fcc P a645dd38cfa74cf586a45ea98a504b1f [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 18:38:18.675743 8938 raft_consensus.cc:482] T a0976eb3149941de9b2d864dc8cd0fcc P a645dd38cfa74cf586a45ea98a504b1f [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 18:38:18.675800 8938 raft_consensus.cc:3037] T a0976eb3149941de9b2d864dc8cd0fcc P a645dd38cfa74cf586a45ea98a504b1f [term 0 FOLLOWER]: Advancing to term 1
I20260806 18:38:18.676841 8938 raft_consensus.cc:504] T a0976eb3149941de9b2d864dc8cd0fcc P a645dd38cfa74cf586a45ea98a504b1f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c6ef26c2af3c4f86bd8359d8258b5ac9" member_type: VOTER last_known_addr { host: "127.19.194.2" port: 42903 } } peers { permanent_uuid: "916b847b199249b297e62dfe70e8ff9d" member_type: VOTER last_known_addr { host: "127.19.194.1" port: 32901 } } peers { permanent_uuid: "a645dd38cfa74cf586a45ea98a504b1f" member_type: VOTER last_known_addr { host: "127.19.194.3" port: 33405 } }
I20260806 18:38:18.677172 8938 leader_election.cc:288] T a0976eb3149941de9b2d864dc8cd0fcc P a645dd38cfa74cf586a45ea98a504b1f [CANDIDATE]: Term 1 election: Requested vote from peers c6ef26c2af3c4f86bd8359d8258b5ac9 (127.19.194.2:42903), 916b847b199249b297e62dfe70e8ff9d (127.19.194.1:32901)
I20260806 18:38:18.677559 8711 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a0976eb3149941de9b2d864dc8cd0fcc" candidate_uuid: "a645dd38cfa74cf586a45ea98a504b1f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c6ef26c2af3c4f86bd8359d8258b5ac9"
I20260806 18:38:18.677580 8570 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a0976eb3149941de9b2d864dc8cd0fcc" candidate_uuid: "a645dd38cfa74cf586a45ea98a504b1f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "916b847b199249b297e62dfe70e8ff9d"
I20260806 18:38:18.677696 8711 raft_consensus.cc:3037] T a0976eb3149941de9b2d864dc8cd0fcc P c6ef26c2af3c4f86bd8359d8258b5ac9 [term 0 FOLLOWER]: Advancing to term 1
I20260806 18:38:18.677695 8570 raft_consensus.cc:3037] T a0976eb3149941de9b2d864dc8cd0fcc P 916b847b199249b297e62dfe70e8ff9d [term 0 FOLLOWER]: Advancing to term 1
I20260806 18:38:18.678962 8711 raft_consensus.cc:2454] T a0976eb3149941de9b2d864dc8cd0fcc P c6ef26c2af3c4f86bd8359d8258b5ac9 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a645dd38cfa74cf586a45ea98a504b1f in term 1.
I20260806 18:38:18.678963 8570 raft_consensus.cc:2454] T a0976eb3149941de9b2d864dc8cd0fcc P 916b847b199249b297e62dfe70e8ff9d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a645dd38cfa74cf586a45ea98a504b1f in term 1.
I20260806 18:38:18.679275 8785 leader_election.cc:302] T a0976eb3149941de9b2d864dc8cd0fcc P a645dd38cfa74cf586a45ea98a504b1f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 916b847b199249b297e62dfe70e8ff9d, a645dd38cfa74cf586a45ea98a504b1f, c6ef26c2af3c4f86bd8359d8258b5ac9; no voters:
I20260806 18:38:18.679514 8938 raft_consensus.cc:2781] T a0976eb3149941de9b2d864dc8cd0fcc P a645dd38cfa74cf586a45ea98a504b1f [term 1 FOLLOWER]: Leader election won for term 1
I20260806 18:38:18.679757 8938 raft_consensus.cc:686] T a0976eb3149941de9b2d864dc8cd0fcc P a645dd38cfa74cf586a45ea98a504b1f [term 1 LEADER]: Becoming Leader. State: Replica: a645dd38cfa74cf586a45ea98a504b1f, State: Running, Role: LEADER
I20260806 18:38:18.680112 8938 consensus_queue.cc:237] T a0976eb3149941de9b2d864dc8cd0fcc P a645dd38cfa74cf586a45ea98a504b1f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c6ef26c2af3c4f86bd8359d8258b5ac9" member_type: VOTER last_known_addr { host: "127.19.194.2" port: 42903 } } peers { permanent_uuid: "916b847b199249b297e62dfe70e8ff9d" member_type: VOTER last_known_addr { host: "127.19.194.1" port: 32901 } } peers { permanent_uuid: "a645dd38cfa74cf586a45ea98a504b1f" member_type: VOTER last_known_addr { host: "127.19.194.3" port: 33405 } }
I20260806 18:38:18.683229 8420 catalog_manager.cc:5495] T a0976eb3149941de9b2d864dc8cd0fcc P a645dd38cfa74cf586a45ea98a504b1f reported cstate change: term changed from 0 to 1, leader changed from <none> to a645dd38cfa74cf586a45ea98a504b1f (127.19.194.3). New cstate: current_term: 1 leader_uuid: "a645dd38cfa74cf586a45ea98a504b1f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c6ef26c2af3c4f86bd8359d8258b5ac9" member_type: VOTER last_known_addr { host: "127.19.194.2" port: 42903 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "916b847b199249b297e62dfe70e8ff9d" member_type: VOTER last_known_addr { host: "127.19.194.1" port: 32901 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a645dd38cfa74cf586a45ea98a504b1f" member_type: VOTER last_known_addr { host: "127.19.194.3" port: 33405 } health_report { overall_health: HEALTHY } } }
I20260806 18:38:18.724711 8927 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:38:18.721241 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51738 (local address 127.19.194.3:33405)
0806 18:38:18.721384 (+ 143us) server_negotiation.cc:206] Beginning negotiation
0806 18:38:18.721387 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 18:38:18.721400 (+ 13us) server_negotiation.cc:407] Connection header received
0806 18:38:18.721535 (+ 135us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:38:18.721537 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:38:18.721592 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:38:18.721680 (+ 88us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 18:38:18.721888 (+ 208us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:38:18.722373 (+ 485us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:38:18.723405 (+ 1032us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:38:18.723615 (+ 210us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:38:18.723716 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:38:18.724031 (+ 315us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 18:38:18.724424 (+ 393us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 18:38:18.724512 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0806 18:38:18.724570 (+ 58us) server_negotiation.cc:299] Negotiation successful
0806 18:38:18.724614 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260806 18:38:18.730377 8711 raft_consensus.cc:1261] T a0976eb3149941de9b2d864dc8cd0fcc P c6ef26c2af3c4f86bd8359d8258b5ac9 [term 1 FOLLOWER]: Refusing update from remote peer a645dd38cfa74cf586a45ea98a504b1f: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 18:38:18.730373 8570 raft_consensus.cc:1261] T a0976eb3149941de9b2d864dc8cd0fcc P 916b847b199249b297e62dfe70e8ff9d [term 1 FOLLOWER]: Refusing update from remote peer a645dd38cfa74cf586a45ea98a504b1f: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 18:38:18.731053 8938 consensus_queue.cc:1035] T a0976eb3149941de9b2d864dc8cd0fcc P a645dd38cfa74cf586a45ea98a504b1f [LEADER]: Connected to new peer: Peer: permanent_uuid: "916b847b199249b297e62dfe70e8ff9d" member_type: VOTER last_known_addr { host: "127.19.194.1" port: 32901 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 18:38:18.731236 8940 consensus_queue.cc:1035] T a0976eb3149941de9b2d864dc8cd0fcc P a645dd38cfa74cf586a45ea98a504b1f [LEADER]: Connected to new peer: Peer: permanent_uuid: "c6ef26c2af3c4f86bd8359d8258b5ac9" member_type: VOTER last_known_addr { host: "127.19.194.2" port: 42903 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 18:38:18.739313 8945 mvcc.cc:205] Tried to move back new op lower bound from 7315625978792824832 to 7315625978596360192. Current Snapshot: MvccSnapshot[applied={T|T < 7315625978792824832}]
I20260806 18:38:18.740588 8944 mvcc.cc:205] Tried to move back new op lower bound from 7315625978792824832 to 7315625978596360192. Current Snapshot: MvccSnapshot[applied={T|T < 7315625978792824832}]
I20260806 18:38:18.749019 8421 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:56694:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 18:38:18.749176 8421 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:56694:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 18:38:18.751518 8421 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 5a2debea42ec454c8177fa1dc84c772f: Sending DeleteTablet for 3 replicas of tablet a0976eb3149941de9b2d864dc8cd0fcc
I20260806 18:38:18.752293 8689 tablet_service.cc:1514] Processing DeleteTablet for tablet a0976eb3149941de9b2d864dc8cd0fcc with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 18:38:18 UTC) from {username='slave'} at 127.0.0.1:38482
I20260806 18:38:18.752476 8834 tablet_service.cc:1514] Processing DeleteTablet for tablet a0976eb3149941de9b2d864dc8cd0fcc with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 18:38:18 UTC) from {username='slave'} at 127.0.0.1:51726
I20260806 18:38:18.752761 8549 tablet_service.cc:1514] Processing DeleteTablet for tablet a0976eb3149941de9b2d864dc8cd0fcc with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 18:38:18 UTC) from {username='slave'} at 127.0.0.1:60238
I20260806 18:38:18.753015 8954 tablet_replica.cc:330] T a0976eb3149941de9b2d864dc8cd0fcc P a645dd38cfa74cf586a45ea98a504b1f: stopping tablet replica
I20260806 18:38:18.753031 20232 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJW69tn/build/debug/bin/kudu with pid 8484
I20260806 18:38:18.753311 8954 raft_consensus.cc:2229] T a0976eb3149941de9b2d864dc8cd0fcc P a645dd38cfa74cf586a45ea98a504b1f [term 1 LEADER]: Raft consensus shutting down.
I20260806 18:38:18.753350 8953 tablet_replica.cc:330] T a0976eb3149941de9b2d864dc8cd0fcc P c6ef26c2af3c4f86bd8359d8258b5ac9: stopping tablet replica
I20260806 18:38:18.753756 8953 raft_consensus.cc:2229] T a0976eb3149941de9b2d864dc8cd0fcc P c6ef26c2af3c4f86bd8359d8258b5ac9 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 18:38:18.753782 8954 raft_consensus.cc:2258] T a0976eb3149941de9b2d864dc8cd0fcc P a645dd38cfa74cf586a45ea98a504b1f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 18:38:18.754565 8953 raft_consensus.cc:2258] T a0976eb3149941de9b2d864dc8cd0fcc P c6ef26c2af3c4f86bd8359d8258b5ac9 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 18:38:18.756108 8953 ts_tablet_manager.cc:1905] T a0976eb3149941de9b2d864dc8cd0fcc P c6ef26c2af3c4f86bd8359d8258b5ac9: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 18:38:18.756184 8954 ts_tablet_manager.cc:1905] T a0976eb3149941de9b2d864dc8cd0fcc P a645dd38cfa74cf586a45ea98a504b1f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 18:38:18.759043 8953 ts_tablet_manager.cc:1918] T a0976eb3149941de9b2d864dc8cd0fcc P c6ef26c2af3c4f86bd8359d8258b5ac9: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 18:38:18.759244 8953 log.cc:1196] T a0976eb3149941de9b2d864dc8cd0fcc P c6ef26c2af3c4f86bd8359d8258b5ac9: Deleting WAL directory at /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786041246094603-20232-0/minicluster-data/ts-1/wal/wals/a0976eb3149941de9b2d864dc8cd0fcc
I20260806 18:38:18.759690 8953 ts_tablet_manager.cc:1939] T a0976eb3149941de9b2d864dc8cd0fcc P c6ef26c2af3c4f86bd8359d8258b5ac9: Deleting consensus metadata
I20260806 18:38:18.760413 8954 ts_tablet_manager.cc:1918] T a0976eb3149941de9b2d864dc8cd0fcc P a645dd38cfa74cf586a45ea98a504b1f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 18:38:18.760619 8954 log.cc:1196] T a0976eb3149941de9b2d864dc8cd0fcc P a645dd38cfa74cf586a45ea98a504b1f: Deleting WAL directory at /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786041246094603-20232-0/minicluster-data/ts-2/wal/wals/a0976eb3149941de9b2d864dc8cd0fcc
I20260806 18:38:18.760944 8406 catalog_manager.cc:4841] TS c6ef26c2af3c4f86bd8359d8258b5ac9 (127.19.194.2:42903): tablet a0976eb3149941de9b2d864dc8cd0fcc (table test-table [id=29c052cb96f449ca82f731e87cedeb75]) successfully deleted
I20260806 18:38:18.761071 8954 ts_tablet_manager.cc:1939] T a0976eb3149941de9b2d864dc8cd0fcc P a645dd38cfa74cf586a45ea98a504b1f: Deleting consensus metadata
I20260806 18:38:18.762373 8407 catalog_manager.cc:4841] TS a645dd38cfa74cf586a45ea98a504b1f (127.19.194.3:33405): tablet a0976eb3149941de9b2d864dc8cd0fcc (table test-table [id=29c052cb96f449ca82f731e87cedeb75]) successfully deleted
W20260806 18:38:18.770215 8409 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.19.194.1:32901 (error 108)
I20260806 18:38:18.770390 20232 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJW69tn/build/debug/bin/kudu with pid 8623
I20260806 18:38:18.771983 8921 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:38:18.771589 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.194.1:32901 (local address 127.0.0.1:60252)
0806 18:38:18.771786 (+ 197us) negotiation.cc:107] Waiting for socket to connect
0806 18:38:18.771865 (+ 79us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.194.1:32901: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":101}
W20260806 18:38:18.772303 8409 catalog_manager.cc:4584] TS 916b847b199249b297e62dfe70e8ff9d (127.19.194.1:32901): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet a0976eb3149941de9b2d864dc8cd0fcc: Network error: Client connection negotiation failed: client connection to 127.19.194.1:32901: connect: Connection refused (error 111)
I20260806 18:38:18.777719 20232 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJW69tn/build/debug/bin/kudu with pid 8763
I20260806 18:38:18.783898 20232 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJW69tn/build/debug/bin/kudu with pid 8390
2026-08-06T18:38:18Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/5 (1099 ms)
[----------] 6 tests from EncryptionPolicyTest (10425 ms total)
[----------] 18 tests from AuthTokenIssuingTest
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/0
Loading random data
Initializing database '/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786041246094603-20232-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 18:38:18 dist-test-slave-1xrh krb5kdc[8960](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 18:38:18 dist-test-slave-1xrh krb5kdc[8960](info): set up 2 sockets
Aug 06 18:38:18 dist-test-slave-1xrh krb5kdc[8960](info): commencing operation
krb5kdc: starting...
W20260806 18:38:19.833622 20232 mini_kdc.cc:121] Time spent starting KDC: real 1.027s user 0.001s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786041246094603-20232-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786041246094603-20232-0/krb5kdc/test-user.keytab.
Aug 06 18:38:19 dist-test-slave-1xrh krb5kdc[8960](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041499, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T18:38:19Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T18:38:19Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786041246094603-20232-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786041246094603-20232-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260806 18:38:19.953742 20232 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJW69tn/build/debug/bin/kudu
/tmp/dist-test-taskJW69tn/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786041246094603-20232-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786041246094603-20232-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786041246094603-20232-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786041246094603-20232-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:36605
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786041246094603-20232-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:40081
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:36605
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786041246094603-20232-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786041246094603-20232-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786041246094603-20232-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 18:38:20.063865 8976 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 18:38:20.064126 8976 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 18:38:20.064196 8976 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 18:38:20.068051 8976 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 18:38:20.068101 8976 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 18:38:20.068120 8976 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 18:38:20.068136 8976 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 18:38:20.068178 8976 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 18:38:20.073310 8976 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:40081
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786041246094603-20232-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786041246094603-20232-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:36605
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786041246094603-20232-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:36605
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786041246094603-20232-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8976
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786041246094603-20232-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 17:59:13 UTC on ccycloud.quasar-wvsnzx.root.comops.site
build id 4571028
I20260806 18:38:20.074396 8976 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 18:38:20.075476 8976 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 18:38:20.083668 8987 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:38:20.083683 8985 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 18:38:20.083889 8976 server_base.cc:1034] running on GCE node
W20260806 18:38:20.084033 8984 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 18:38:20.084554 8976 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 18:38:20.085618 8976 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 18:38:20.086804 8976 hybrid_clock.cc:648] HybridClock initialized: now 1786041500086782 us; error 38 us; skew 500 ppm
Aug 06 18:38:20 dist-test-slave-1xrh krb5kdc[8960](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041500, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 18:38:20.089453 8976 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260806 18:38:20.090308 8976 webserver.cc:460] Webserver started at http://127.0.0.1:39299/ using document root <none> and password file <none>
I20260806 18:38:20.090817 8976 fs_manager.cc:359] Metadata directory not provided
I20260806 18:38:20.090903 8976 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 18:38:20.091116 8976 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 18:38:20.093091 8976 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786041246094603-20232-0/minicluster-data/master-0/data/instance:
uuid: "49823037067147f7b1c79a61e3b4e9e6"
format_stamp: "Formatted at 2026-08-06 18:38:20 on dist-test-slave-1xrh"
server_key: "ea63303536dbe82300279b089a15f806"
server_key_iv: "71ab623ee2167297f7f7ef40c35327ad"
server_key_version: "encryptionkey@0"
I20260806 18:38:20.093632 8976 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786041246094603-20232-0/minicluster-data/master-0/wal/instance:
uuid: "49823037067147f7b1c79a61e3b4e9e6"
format_stamp: "Formatted at 2026-08-06 18:38:20 on dist-test-slave-1xrh"
server_key: "ea63303536dbe82300279b089a15f806"
server_key_iv: "71ab623ee2167297f7f7ef40c35327ad"
server_key_version: "encryptionkey@0"
I20260806 18:38:20.097080 8976 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.005s sys 0.000s
I20260806 18:38:20.099325 8995 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 18:38:20.100361 8976 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260806 18:38:20.100488 8976 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786041246094603-20232-0/minicluster-data/master-0/data,/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786041246094603-20232-0/minicluster-data/master-0/wal
uuid: "49823037067147f7b1c79a61e3b4e9e6"
format_stamp: "Formatted at 2026-08-06 18:38:20 on dist-test-slave-1xrh"
server_key: "ea63303536dbe82300279b089a15f806"
server_key_iv: "71ab623ee2167297f7f7ef40c35327ad"
server_key_version: "encryptionkey@0"
I20260806 18:38:20.100587 8976 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786041246094603-20232-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786041246094603-20232-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786041246094603-20232-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 18:38:20.128237 8976 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 18:38:20.131080 8976 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 18:38:20.131305 8976 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 18:38:20.139062 8976 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:36605
I20260806 18:38:20.139074 9054 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:36605 every 8 connection(s)
I20260806 18:38:20.140196 8976 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786041246094603-20232-0/minicluster-data/master-0/data/info.pb
I20260806 18:38:20.142838 9055 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 18:38:20.148068 9055 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 49823037067147f7b1c79a61e3b4e9e6: Bootstrap starting.
I20260806 18:38:20.149860 20232 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJW69tn/build/debug/bin/kudu as pid 8976
I20260806 18:38:20.150025 20232 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786041246094603-20232-0/minicluster-data/master-0/wal/instance
I20260806 18:38:20.150334 20232 external_mini_cluster.cc:1428] Setting key c0491a1f1cf1c2092a0db122b03fd22c
I20260806 18:38:20.150398 9055 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 49823037067147f7b1c79a61e3b4e9e6: Neither blocks nor log segments found. Creating new log.
I20260806 18:38:20.151104 9055 log.cc:824] T 00000000000000000000000000000000 P 49823037067147f7b1c79a61e3b4e9e6: Log is configured to *not* fsync() on all Append() calls
I20260806 18:38:20.153227 9055 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 49823037067147f7b1c79a61e3b4e9e6: No bootstrap required, opened a new log
I20260806 18:38:20.157797 9055 raft_consensus.cc:348] T 00000000000000000000000000000000 P 49823037067147f7b1c79a61e3b4e9e6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "49823037067147f7b1c79a61e3b4e9e6" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36605 } }
I20260806 18:38:20.158094 9055 raft_consensus.cc:374] T 00000000000000000000000000000000 P 49823037067147f7b1c79a61e3b4e9e6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 18:38:20.158159 9055 raft_consensus.cc:729] T 00000000000000000000000000000000 P 49823037067147f7b1c79a61e3b4e9e6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 49823037067147f7b1c79a61e3b4e9e6, State: Initialized, Role: FOLLOWER
I20260806 18:38:20.158696 9055 consensus_queue.cc:260] T 00000000000000000000000000000000 P 49823037067147f7b1c79a61e3b4e9e6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "49823037067147f7b1c79a61e3b4e9e6" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36605 } }
I20260806 18:38:20.158913 9055 raft_consensus.cc:388] T 00000000000000000000000000000000 P 49823037067147f7b1c79a61e3b4e9e6 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 18:38:20.159015 9055 raft_consensus.cc:482] T 00000000000000000000000000000000 P 49823037067147f7b1c79a61e3b4e9e6 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 18:38:20.159139 9055 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 49823037067147f7b1c79a61e3b4e9e6 [term 0 FOLLOWER]: Advancing to term 1
Aug 06 18:38:20 dist-test-slave-1xrh krb5kdc[8960](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041499, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260806 18:38:20.160161 9055 raft_consensus.cc:504] T 00000000000000000000000000000000 P 49823037067147f7b1c79a61e3b4e9e6 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "49823037067147f7b1c79a61e3b4e9e6" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36605 } }
I20260806 18:38:20.160523 9055 leader_election.cc:302] T 00000000000000000000000000000000 P 49823037067147f7b1c79a61e3b4e9e6 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 49823037067147f7b1c79a61e3b4e9e6; no voters:
I20260806 18:38:20.160903 9055 leader_election.cc:288] T 00000000000000000000000000000000 P 49823037067147f7b1c79a61e3b4e9e6 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 18:38:20.161031 9061 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 49823037067147f7b1c79a61e3b4e9e6 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 18:38:20.161235 9061 raft_consensus.cc:686] T 00000000000000000000000000000000 P 49823037067147f7b1c79a61e3b4e9e6 [term 1 LEADER]: Becoming Leader. State: Replica: 49823037067147f7b1c79a61e3b4e9e6, State: Running, Role: LEADER
I20260806 18:38:20.161628 9061 consensus_queue.cc:237] T 00000000000000000000000000000000 P 49823037067147f7b1c79a61e3b4e9e6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "49823037067147f7b1c79a61e3b4e9e6" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36605 } }
I20260806 18:38:20.162150 9055 sys_catalog.cc:564] T 00000000000000000000000000000000 P 49823037067147f7b1c79a61e3b4e9e6 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 18:38:20.162763 9061 sys_catalog.cc:455] T 00000000000000000000000000000000 P 49823037067147f7b1c79a61e3b4e9e6 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 49823037067147f7b1c79a61e3b4e9e6. Latest consensus state: current_term: 1 leader_uuid: "49823037067147f7b1c79a61e3b4e9e6" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "49823037067147f7b1c79a61e3b4e9e6" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36605 } } }
I20260806 18:38:20.162887 9061 sys_catalog.cc:458] T 00000000000000000000000000000000 P 49823037067147f7b1c79a61e3b4e9e6 [sys.catalog]: This master's current role is: LEADER
I20260806 18:38:20.162763 9062 sys_catalog.cc:455] T 00000000000000000000000000000000 P 49823037067147f7b1c79a61e3b4e9e6 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "49823037067147f7b1c79a61e3b4e9e6" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "49823037067147f7b1c79a61e3b4e9e6" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36605 } } }
I20260806 18:38:20.163312 9062 sys_catalog.cc:458] T 00000000000000000000000000000000 P 49823037067147f7b1c79a61e3b4e9e6 [sys.catalog]: This master's current role is: LEADER
I20260806 18:38:20.163627 9066 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 18:38:20.166182 9059 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:38:20.151507 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57955 (local address 127.0.0.1:36605)
0806 18:38:20.151861 (+ 354us) server_negotiation.cc:206] Beginning negotiation
0806 18:38:20.151884 (+ 23us) server_negotiation.cc:399] Waiting for connection header
0806 18:38:20.151917 (+ 33us) server_negotiation.cc:407] Connection header received
0806 18:38:20.152509 (+ 592us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:38:20.152524 (+ 15us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:38:20.152828 (+ 304us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:38:20.153089 (+ 261us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:38:20.154111 (+ 1022us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:38:20.155066 (+ 955us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:38:20.156492 (+ 1426us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:38:20.156944 (+ 452us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:38:20.157054 (+ 110us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:38:20.160128 (+ 3074us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:38:20.160133 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:38:20.160145 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 18:38:20.160169 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:38:20.162256 (+ 2087us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:38:20.162944 (+ 688us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:38:20.162953 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:38:20.162961 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:38:20.163130 (+ 169us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:38:20.163674 (+ 544us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:38:20.163678 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:38:20.163679 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:38:20.164340 (+ 661us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:38:20.164602 (+ 262us) server_negotiation.cc:1035] Waiting for connection context
0806 18:38:20.164962 (+ 360us) server_negotiation.cc:299] Negotiation successful
0806 18:38:20.165226 (+ 264us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":216,"thread_start_us":96,"threads_started":1}
I20260806 18:38:20.167493 9066 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 18:38:20.169667 20232 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260806 18:38:20.176416 9066 catalog_manager.cc:1347] Generated new cluster ID: d5e2051c46004351bfa18e30938140f0
I20260806 18:38:20.176528 9066 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 18:38:20.180812 9059 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:38:20.171885 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34751 (local address 127.0.0.1:36605)
0806 18:38:20.171971 (+ 86us) server_negotiation.cc:206] Beginning negotiation
0806 18:38:20.171975 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 18:38:20.172209 (+ 234us) server_negotiation.cc:407] Connection header received
0806 18:38:20.172408 (+ 199us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:38:20.172411 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:38:20.172450 (+ 39us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:38:20.172521 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:38:20.173337 (+ 816us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:38:20.174153 (+ 816us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:38:20.175290 (+ 1137us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:38:20.175675 (+ 385us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:38:20.175749 (+ 74us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:38:20.177522 (+ 1773us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:38:20.177525 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:38:20.177528 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 18:38:20.177552 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:38:20.179092 (+ 1540us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:38:20.179616 (+ 524us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:38:20.179619 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:38:20.179620 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:38:20.179693 (+ 73us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:38:20.180011 (+ 318us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:38:20.180014 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:38:20.180016 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:38:20.180184 (+ 168us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:38:20.180310 (+ 126us) server_negotiation.cc:1035] Waiting for connection context
0806 18:38:20.180594 (+ 284us) server_negotiation.cc:299] Negotiation successful
0806 18:38:20.180676 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260806 18:38:20.186702 9066 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 18:38:20.187820 9066 catalog_manager.cc:1504] Loading token signing keys...
I20260806 18:38:20.205842 9066 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 49823037067147f7b1c79a61e3b4e9e6: Generated new TSK 0
I20260806 18:38:20.206622 9066 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 18:38:20.213907 20232 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJW69tn/build/debug/bin/kudu with pid 8976
W20260806 18:38:20.221330 8958 connection.cc:535] client connection to 127.0.0.1:36605 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:36605): Connection reset by peer (error 104)
2026-08-06T18:38:20Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/0 (1429 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/1
Loading random data
Initializing database '/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786041246094603-20232-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 18:38:20 dist-test-slave-1xrh krb5kdc[9089](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 18:38:20 dist-test-slave-1xrh krb5kdc[9089](info): set up 2 sockets
Aug 06 18:38:20 dist-test-slave-1xrh krb5kdc[9089](info): commencing operation
krb5kdc: starting...
W20260806 18:38:21.291961 20232 mini_kdc.cc:121] Time spent starting KDC: real 1.056s user 0.005s sys 0.001s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786041246094603-20232-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786041246094603-20232-0/krb5kdc/test-user.keytab.
Aug 06 18:38:21 dist-test-slave-1xrh krb5kdc[9089](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041501, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T18:38:21Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T18:38:21Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786041246094603-20232-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786041246094603-20232-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260806 18:38:21.414007 20232 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJW69tn/build/debug/bin/kudu
/tmp/dist-test-taskJW69tn/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786041246094603-20232-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786041246094603-20232-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786041246094603-20232-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786041246094603-20232-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:39667
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786041246094603-20232-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:35755
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:39667
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786041246094603-20232-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786041246094603-20232-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786041246094603-20232-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 18:38:21.521672 9103 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 18:38:21.521914 9103 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 18:38:21.521953 9103 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 18:38:21.525933 9103 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 18:38:21.525987 9103 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 18:38:21.526036 9103 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 18:38:21.526075 9103 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 18:38:21.526110 9103 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 18:38:21.531216 9103 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:35755
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786041246094603-20232-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786041246094603-20232-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:39667
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786041246094603-20232-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:39667
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786041246094603-20232-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9103
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786041246094603-20232-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 17:59:13 UTC on ccycloud.quasar-wvsnzx.root.comops.site
build id 4571028
I20260806 18:38:21.532166 9103 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 18:38:21.533134 9103 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 18:38:21.541401 9110 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:38:21.541456 9108 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 18:38:21.541581 9103 server_base.cc:1034] running on GCE node
W20260806 18:38:21.541419 9112 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 18:38:21.542239 9103 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 18:38:21.543306 9103 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 18:38:21.544481 9103 hybrid_clock.cc:648] HybridClock initialized: now 1786041501544467 us; error 25 us; skew 500 ppm
Aug 06 18:38:21 dist-test-slave-1xrh krb5kdc[9089](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041501, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 18:38:21.547058 9103 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260806 18:38:21.548111 9103 webserver.cc:460] Webserver started at http://127.0.0.1:33861/ using document root <none> and password file <none>
I20260806 18:38:21.548660 9103 fs_manager.cc:359] Metadata directory not provided
I20260806 18:38:21.548744 9103 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 18:38:21.548946 9103 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 18:38:21.550803 9103 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786041246094603-20232-0/minicluster-data/master-0/data/instance:
uuid: "5c226faab069437faffc1a38f0502442"
format_stamp: "Formatted at 2026-08-06 18:38:21 on dist-test-slave-1xrh"
server_key: "802c505ce0d329a73d1bdc83628b62f2"
server_key_iv: "4d95d4b783a097b5f169760bf88fe5f4"
server_key_version: "encryptionkey@0"
I20260806 18:38:21.551388 9103 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786041246094603-20232-0/minicluster-data/master-0/wal/instance:
uuid: "5c226faab069437faffc1a38f0502442"
format_stamp: "Formatted at 2026-08-06 18:38:21 on dist-test-slave-1xrh"
server_key: "802c505ce0d329a73d1bdc83628b62f2"
server_key_iv: "4d95d4b783a097b5f169760bf88fe5f4"
server_key_version: "encryptionkey@0"
I20260806 18:38:21.554786 9103 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.002s sys 0.001s
I20260806 18:38:21.557030 9120 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 18:38:21.558159 9103 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260806 18:38:21.558307 9103 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786041246094603-20232-0/minicluster-data/master-0/data,/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786041246094603-20232-0/minicluster-data/master-0/wal
uuid: "5c226faab069437faffc1a38f0502442"
format_stamp: "Formatted at 2026-08-06 18:38:21 on dist-test-slave-1xrh"
server_key: "802c505ce0d329a73d1bdc83628b62f2"
server_key_iv: "4d95d4b783a097b5f169760bf88fe5f4"
server_key_version: "encryptionkey@0"
I20260806 18:38:21.558408 9103 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786041246094603-20232-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786041246094603-20232-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786041246094603-20232-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 18:38:21.574745 9103 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 18:38:21.577351 9103 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 18:38:21.577560 9103 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 18:38:21.584292 9103 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:39667
I20260806 18:38:21.584352 9176 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:39667 every 8 connection(s)
I20260806 18:38:21.585196 9103 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786041246094603-20232-0/minicluster-data/master-0/data/info.pb
I20260806 18:38:21.587870 9177 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 18:38:21.589864 20232 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJW69tn/build/debug/bin/kudu as pid 9103
I20260806 18:38:21.590010 20232 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786041246094603-20232-0/minicluster-data/master-0/wal/instance
I20260806 18:38:21.590289 20232 external_mini_cluster.cc:1428] Setting key aa067a76caf9038d1731f6a948a148d8
I20260806 18:38:21.593540 9177 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5c226faab069437faffc1a38f0502442: Bootstrap starting.
I20260806 18:38:21.595748 9177 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 5c226faab069437faffc1a38f0502442: Neither blocks nor log segments found. Creating new log.
I20260806 18:38:21.596449 9177 log.cc:824] T 00000000000000000000000000000000 P 5c226faab069437faffc1a38f0502442: Log is configured to *not* fsync() on all Append() calls
I20260806 18:38:21.598277 9177 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5c226faab069437faffc1a38f0502442: No bootstrap required, opened a new log
I20260806 18:38:21.598783 9180 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:38:21.591591 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42821 (local address 127.0.0.1:39667)
0806 18:38:21.592025 (+ 434us) server_negotiation.cc:206] Beginning negotiation
0806 18:38:21.592040 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0806 18:38:21.592072 (+ 32us) server_negotiation.cc:407] Connection header received
0806 18:38:21.592771 (+ 699us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:38:21.592788 (+ 17us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:38:21.593107 (+ 319us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:38:21.593403 (+ 296us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:38:21.593789 (+ 386us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:38:21.594737 (+ 948us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:38:21.595695 (+ 958us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:38:21.596167 (+ 472us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:38:21.596333 (+ 166us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:38:21.597170 (+ 837us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:38:21.597175 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:38:21.597200 (+ 25us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 18:38:21.597556 (+ 356us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:38:21.597606 (+ 50us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 18:38:21.597626 (+ 20us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:38:21.597756 (+ 130us) server_negotiation.cc:1035] Waiting for connection context
0806 18:38:21.597927 (+ 171us) server_negotiation.cc:299] Negotiation successful
0806 18:38:21.598054 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":302,"thread_start_us":134,"threads_started":1}
I20260806 18:38:21.601508 20232 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260806 18:38:21.602049 9177 raft_consensus.cc:348] T 00000000000000000000000000000000 P 5c226faab069437faffc1a38f0502442 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5c226faab069437faffc1a38f0502442" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39667 } }
I20260806 18:38:21.602270 9177 raft_consensus.cc:374] T 00000000000000000000000000000000 P 5c226faab069437faffc1a38f0502442 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 18:38:21.602355 9177 raft_consensus.cc:729] T 00000000000000000000000000000000 P 5c226faab069437faffc1a38f0502442 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5c226faab069437faffc1a38f0502442, State: Initialized, Role: FOLLOWER
I20260806 18:38:21.602792 9177 consensus_queue.cc:260] T 00000000000000000000000000000000 P 5c226faab069437faffc1a38f0502442 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5c226faab069437faffc1a38f0502442" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39667 } }
I20260806 18:38:21.602941 9177 raft_consensus.cc:388] T 00000000000000000000000000000000 P 5c226faab069437faffc1a38f0502442 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 18:38:21.603008 9177 raft_consensus.cc:482] T 00000000000000000000000000000000 P 5c226faab069437faffc1a38f0502442 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 18:38:21.603102 9177 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 5c226faab069437faffc1a38f0502442 [term 0 FOLLOWER]: Advancing to term 1
I20260806 18:38:21.604206 9177 raft_consensus.cc:504] T 00000000000000000000000000000000 P 5c226faab069437faffc1a38f0502442 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5c226faab069437faffc1a38f0502442" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39667 } }
I20260806 18:38:21.604564 9177 leader_election.cc:302] T 00000000000000000000000000000000 P 5c226faab069437faffc1a38f0502442 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 5c226faab069437faffc1a38f0502442; no voters:
I20260806 18:38:21.604878 9177 leader_election.cc:288] T 00000000000000000000000000000000 P 5c226faab069437faffc1a38f0502442 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 18:38:21.604952 9187 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 5c226faab069437faffc1a38f0502442 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 18:38:21.605158 9187 raft_consensus.cc:686] T 00000000000000000000000000000000 P 5c226faab069437faffc1a38f0502442 [term 1 LEADER]: Becoming Leader. State: Replica: 5c226faab069437faffc1a38f0502442, State: Running, Role: LEADER
I20260806 18:38:21.605482 9187 consensus_queue.cc:237] T 00000000000000000000000000000000 P 5c226faab069437faffc1a38f0502442 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5c226faab069437faffc1a38f0502442" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39667 } }
I20260806 18:38:21.606108 9177 sys_catalog.cc:564] T 00000000000000000000000000000000 P 5c226faab069437faffc1a38f0502442 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 18:38:21.606657 9187 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5c226faab069437faffc1a38f0502442 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 5c226faab069437faffc1a38f0502442. Latest consensus state: current_term: 1 leader_uuid: "5c226faab069437faffc1a38f0502442" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5c226faab069437faffc1a38f0502442" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39667 } } }
I20260806 18:38:21.606773 9187 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5c226faab069437faffc1a38f0502442 [sys.catalog]: This master's current role is: LEADER
I20260806 18:38:21.607589 9188 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5c226faab069437faffc1a38f0502442 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "5c226faab069437faffc1a38f0502442" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5c226faab069437faffc1a38f0502442" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39667 } } }
I20260806 18:38:21.607755 9188 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5c226faab069437faffc1a38f0502442 [sys.catalog]: This master's current role is: LEADER
I20260806 18:38:21.607901 9180 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:38:21.603630 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39063 (local address 127.0.0.1:39667)
0806 18:38:21.603740 (+ 110us) server_negotiation.cc:206] Beginning negotiation
0806 18:38:21.603744 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 18:38:21.603905 (+ 161us) server_negotiation.cc:407] Connection header received
0806 18:38:21.604017 (+ 112us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:38:21.604019 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:38:21.604052 (+ 33us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:38:21.604115 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:38:21.604446 (+ 331us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:38:21.605086 (+ 640us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:38:21.606061 (+ 975us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:38:21.606441 (+ 380us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:38:21.606535 (+ 94us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:38:21.607370 (+ 835us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:38:21.607373 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:38:21.607376 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 18:38:21.607388 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:38:21.607420 (+ 32us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 18:38:21.607436 (+ 16us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:38:21.607529 (+ 93us) server_negotiation.cc:1035] Waiting for connection context
0806 18:38:21.607659 (+ 130us) server_negotiation.cc:299] Negotiation successful
0806 18:38:21.607690 (+ 31us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260806 18:38:21.608170 9196 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 18:38:21.611248 9196 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 18:38:21.616134 9196 catalog_manager.cc:1347] Generated new cluster ID: fb87f6fff83148f39ed3e962a963e4a4
I20260806 18:38:21.616202 9196 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 18:38:21.630620 9196 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 18:38:21.631803 9196 catalog_manager.cc:1504] Loading token signing keys...
I20260806 18:38:21.649652 9196 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 5c226faab069437faffc1a38f0502442: Generated new TSK 0
I20260806 18:38:21.650241 9196 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 18:38:21.658114 20232 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJW69tn/build/debug/bin/kudu with pid 9103
W20260806 18:38:21.665375 9087 connection.cc:535] client connection to 127.0.0.1:39667 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:39667): Connection reset by peer (error 104)
2026-08-06T18:38:21Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/1 (1443 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/2
Loading random data
Initializing database '/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786041246094603-20232-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 18:38:21 dist-test-slave-1xrh krb5kdc[9209](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 18:38:21 dist-test-slave-1xrh krb5kdc[9209](info): set up 2 sockets
Aug 06 18:38:21 dist-test-slave-1xrh krb5kdc[9209](info): commencing operation
krb5kdc: starting...
W20260806 18:38:22.732266 20232 mini_kdc.cc:121] Time spent starting KDC: real 1.053s user 0.003s sys 0.003s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786041246094603-20232-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786041246094603-20232-0/krb5kdc/test-user.keytab.
Aug 06 18:38:22 dist-test-slave-1xrh krb5kdc[9209](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041502, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T18:38:22Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T18:38:22Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786041246094603-20232-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786041246094603-20232-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260806 18:38:22.850365 20232 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJW69tn/build/debug/bin/kudu
/tmp/dist-test-taskJW69tn/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786041246094603-20232-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786041246094603-20232-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786041246094603-20232-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786041246094603-20232-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:36753
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786041246094603-20232-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:41023
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:36753
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786041246094603-20232-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786041246094603-20232-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786041246094603-20232-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 18:38:22.957497 9222 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 18:38:22.957746 9222 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 18:38:22.957785 9222 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 18:38:22.961635 9222 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 18:38:22.961690 9222 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 18:38:22.961714 9222 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 18:38:22.961730 9222 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 18:38:22.961745 9222 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 18:38:22.966936 9222 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:41023
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786041246094603-20232-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786041246094603-20232-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:36753
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786041246094603-20232-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:36753
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786041246094603-20232-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9222
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786041246094603-20232-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 17:59:13 UTC on ccycloud.quasar-wvsnzx.root.comops.site
build id 4571028
I20260806 18:38:22.967913 9222 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 18:38:22.968822 9222 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 18:38:22.977155 9229 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:38:22.977133 9228 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 18:38:22.977212 9222 server_base.cc:1034] running on GCE node
W20260806 18:38:22.977159 9231 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 18:38:22.977900 9222 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 18:38:22.979017 9222 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 18:38:22.980222 9222 hybrid_clock.cc:648] HybridClock initialized: now 1786041502980192 us; error 39 us; skew 500 ppm
Aug 06 18:38:22 dist-test-slave-1xrh krb5kdc[9209](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041502, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 18:38:22.983053 9222 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260806 18:38:22.984004 9222 webserver.cc:460] Webserver started at http://127.0.0.1:35385/ using document root <none> and password file <none>
I20260806 18:38:22.984515 9222 fs_manager.cc:359] Metadata directory not provided
I20260806 18:38:22.984583 9222 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 18:38:22.984747 9222 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 18:38:22.986863 9222 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786041246094603-20232-0/minicluster-data/master-0/data/instance:
uuid: "70eda35666c2400389bedd2e22f23634"
format_stamp: "Formatted at 2026-08-06 18:38:22 on dist-test-slave-1xrh"
server_key: "7939d3e987005750fcde5a255a97eaa5"
server_key_iv: "684ba3f90cf48d67b09f0ebd125692c5"
server_key_version: "encryptionkey@0"
I20260806 18:38:22.987419 9222 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786041246094603-20232-0/minicluster-data/master-0/wal/instance:
uuid: "70eda35666c2400389bedd2e22f23634"
format_stamp: "Formatted at 2026-08-06 18:38:22 on dist-test-slave-1xrh"
server_key: "7939d3e987005750fcde5a255a97eaa5"
server_key_iv: "684ba3f90cf48d67b09f0ebd125692c5"
server_key_version: "encryptionkey@0"
I20260806 18:38:22.991117 9222 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.000s
I20260806 18:38:22.993501 9238 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 18:38:22.994668 9222 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260806 18:38:22.994804 9222 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786041246094603-20232-0/minicluster-data/master-0/data,/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786041246094603-20232-0/minicluster-data/master-0/wal
uuid: "70eda35666c2400389bedd2e22f23634"
format_stamp: "Formatted at 2026-08-06 18:38:22 on dist-test-slave-1xrh"
server_key: "7939d3e987005750fcde5a255a97eaa5"
server_key_iv: "684ba3f90cf48d67b09f0ebd125692c5"
server_key_version: "encryptionkey@0"
I20260806 18:38:22.994944 9222 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786041246094603-20232-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786041246094603-20232-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786041246094603-20232-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 18:38:22.999337 9222 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 18:38:23.002506 9222 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 18:38:23.002647 9222 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 18:38:23.009873 9222 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:36753
I20260806 18:38:23.009900 9291 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:36753 every 8 connection(s)
I20260806 18:38:23.010782 9222 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786041246094603-20232-0/minicluster-data/master-0/data/info.pb
I20260806 18:38:23.013316 9292 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 18:38:23.015458 20232 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJW69tn/build/debug/bin/kudu as pid 9222
I20260806 18:38:23.015597 20232 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786041246094603-20232-0/minicluster-data/master-0/wal/instance
I20260806 18:38:23.015908 20232 external_mini_cluster.cc:1428] Setting key 5313f9c3ad2a7d7ad6f4700f70bdc08f
I20260806 18:38:23.019399 9292 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 70eda35666c2400389bedd2e22f23634: Bootstrap starting.
I20260806 18:38:23.021139 9296 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:38:23.017258 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42715 (local address 127.0.0.1:36753)
0806 18:38:23.017699 (+ 441us) server_negotiation.cc:206] Beginning negotiation
0806 18:38:23.017712 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0806 18:38:23.017747 (+ 35us) server_negotiation.cc:407] Connection header received
0806 18:38:23.018519 (+ 772us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:38:23.018536 (+ 17us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:38:23.018799 (+ 263us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:38:23.019146 (+ 347us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:38:23.019935 (+ 789us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:38:23.019940 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:38:23.019951 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 18:38:23.019961 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:38:23.020010 (+ 49us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 18:38:23.020028 (+ 18us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:38:23.020174 (+ 146us) server_negotiation.cc:1035] Waiting for connection context
0806 18:38:23.020244 (+ 70us) server_negotiation.cc:299] Negotiation successful
0806 18:38:23.020395 (+ 151us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":296,"thread_start_us":153,"threads_started":1}
I20260806 18:38:23.021705 9292 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 70eda35666c2400389bedd2e22f23634: Neither blocks nor log segments found. Creating new log.
I20260806 18:38:23.022410 9292 log.cc:824] T 00000000000000000000000000000000 P 70eda35666c2400389bedd2e22f23634: Log is configured to *not* fsync() on all Append() calls
I20260806 18:38:23.023940 20232 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260806 18:38:23.024504 9292 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 70eda35666c2400389bedd2e22f23634: No bootstrap required, opened a new log
I20260806 18:38:23.028065 9296 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:38:23.026006 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51225 (local address 127.0.0.1:36753)
0806 18:38:23.026106 (+ 100us) server_negotiation.cc:206] Beginning negotiation
0806 18:38:23.026110 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 18:38:23.026411 (+ 301us) server_negotiation.cc:407] Connection header received
0806 18:38:23.026589 (+ 178us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:38:23.026595 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:38:23.026633 (+ 38us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:38:23.026710 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:38:23.027562 (+ 852us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:38:23.027567 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:38:23.027569 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 18:38:23.027583 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:38:23.027618 (+ 35us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 18:38:23.027632 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:38:23.027692 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0806 18:38:23.027898 (+ 206us) server_negotiation.cc:299] Negotiation successful
0806 18:38:23.027942 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260806 18:38:23.029114 9292 raft_consensus.cc:348] T 00000000000000000000000000000000 P 70eda35666c2400389bedd2e22f23634 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "70eda35666c2400389bedd2e22f23634" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36753 } }
I20260806 18:38:23.029439 9292 raft_consensus.cc:374] T 00000000000000000000000000000000 P 70eda35666c2400389bedd2e22f23634 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 18:38:23.029516 9292 raft_consensus.cc:729] T 00000000000000000000000000000000 P 70eda35666c2400389bedd2e22f23634 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 70eda35666c2400389bedd2e22f23634, State: Initialized, Role: FOLLOWER
I20260806 18:38:23.029968 9292 consensus_queue.cc:260] T 00000000000000000000000000000000 P 70eda35666c2400389bedd2e22f23634 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "70eda35666c2400389bedd2e22f23634" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36753 } }
I20260806 18:38:23.030141 9292 raft_consensus.cc:388] T 00000000000000000000000000000000 P 70eda35666c2400389bedd2e22f23634 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 18:38:23.030213 9292 raft_consensus.cc:482] T 00000000000000000000000000000000 P 70eda35666c2400389bedd2e22f23634 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 18:38:23.030320 9292 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 70eda35666c2400389bedd2e22f23634 [term 0 FOLLOWER]: Advancing to term 1
I20260806 18:38:23.031241 9292 raft_consensus.cc:504] T 00000000000000000000000000000000 P 70eda35666c2400389bedd2e22f23634 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "70eda35666c2400389bedd2e22f23634" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36753 } }
I20260806 18:38:23.031570 9292 leader_election.cc:302] T 00000000000000000000000000000000 P 70eda35666c2400389bedd2e22f23634 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 70eda35666c2400389bedd2e22f23634; no voters:
I20260806 18:38:23.031860 9292 leader_election.cc:288] T 00000000000000000000000000000000 P 70eda35666c2400389bedd2e22f23634 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 18:38:23.031960 9304 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 70eda35666c2400389bedd2e22f23634 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 18:38:23.032218 9304 raft_consensus.cc:686] T 00000000000000000000000000000000 P 70eda35666c2400389bedd2e22f23634 [term 1 LEADER]: Becoming Leader. State: Replica: 70eda35666c2400389bedd2e22f23634, State: Running, Role: LEADER
I20260806 18:38:23.032505 9304 consensus_queue.cc:237] T 00000000000000000000000000000000 P 70eda35666c2400389bedd2e22f23634 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "70eda35666c2400389bedd2e22f23634" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36753 } }
I20260806 18:38:23.033025 9292 sys_catalog.cc:564] T 00000000000000000000000000000000 P 70eda35666c2400389bedd2e22f23634 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 18:38:23.034089 9306 sys_catalog.cc:455] T 00000000000000000000000000000000 P 70eda35666c2400389bedd2e22f23634 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 70eda35666c2400389bedd2e22f23634. Latest consensus state: current_term: 1 leader_uuid: "70eda35666c2400389bedd2e22f23634" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "70eda35666c2400389bedd2e22f23634" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36753 } } }
I20260806 18:38:23.034231 9306 sys_catalog.cc:458] T 00000000000000000000000000000000 P 70eda35666c2400389bedd2e22f23634 [sys.catalog]: This master's current role is: LEADER
I20260806 18:38:23.034643 9313 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 18:38:23.035640 9305 sys_catalog.cc:455] T 00000000000000000000000000000000 P 70eda35666c2400389bedd2e22f23634 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "70eda35666c2400389bedd2e22f23634" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "70eda35666c2400389bedd2e22f23634" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36753 } } }
I20260806 18:38:23.035743 9305 sys_catalog.cc:458] T 00000000000000000000000000000000 P 70eda35666c2400389bedd2e22f23634 [sys.catalog]: This master's current role is: LEADER
I20260806 18:38:23.037315 9313 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 18:38:23.042796 9313 catalog_manager.cc:1347] Generated new cluster ID: 8dc3e0033ecc430fbcbc92fa21c864fc
I20260806 18:38:23.042858 9313 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 18:38:23.052546 9313 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 18:38:23.052843 9313 catalog_manager.cc:1504] Loading token signing keys...
I20260806 18:38:23.066046 9313 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 70eda35666c2400389bedd2e22f23634: Generated new TSK 0
I20260806 18:38:23.066756 9313 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 18:38:23.072645 20232 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJW69tn/build/debug/bin/kudu with pid 9222
2026-08-06T18:38:23Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/2 (1414 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/3
Loading random data
Initializing database '/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786041246094603-20232-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 18:38:23 dist-test-slave-1xrh krb5kdc[9325](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 18:38:23 dist-test-slave-1xrh krb5kdc[9325](info): set up 2 sockets
Aug 06 18:38:23 dist-test-slave-1xrh krb5kdc[9325](info): commencing operation
krb5kdc: starting...
W20260806 18:38:24.140354 20232 mini_kdc.cc:121] Time spent starting KDC: real 1.047s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786041246094603-20232-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786041246094603-20232-0/krb5kdc/test-user.keytab.
Aug 06 18:38:24 dist-test-slave-1xrh krb5kdc[9325](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041504, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T18:38:24Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T18:38:24Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786041246094603-20232-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786041246094603-20232-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260806 18:38:24.257920 20232 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJW69tn/build/debug/bin/kudu
/tmp/dist-test-taskJW69tn/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786041246094603-20232-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786041246094603-20232-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786041246094603-20232-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786041246094603-20232-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:46723
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786041246094603-20232-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:36789
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:46723
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786041246094603-20232-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786041246094603-20232-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786041246094603-20232-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 18:38:24.365721 9338 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 18:38:24.365988 9338 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 18:38:24.366030 9338 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 18:38:24.370141 9338 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 18:38:24.370198 9338 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 18:38:24.370216 9338 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 18:38:24.370234 9338 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 18:38:24.370249 9338 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 18:38:24.375751 9338 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:36789
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786041246094603-20232-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786041246094603-20232-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:46723
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786041246094603-20232-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:46723
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786041246094603-20232-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9338
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786041246094603-20232-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 17:59:13 UTC on ccycloud.quasar-wvsnzx.root.comops.site
build id 4571028
I20260806 18:38:24.376710 9338 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 18:38:24.377637 9338 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 18:38:24.385692 9346 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 18:38:24.385789 9338 server_base.cc:1034] running on GCE node
W20260806 18:38:24.385694 9344 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:38:24.385716 9343 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 18:38:24.386552 9338 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 18:38:24.387621 9338 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 18:38:24.388778 9338 hybrid_clock.cc:648] HybridClock initialized: now 1786041504388747 us; error 43 us; skew 500 ppm
Aug 06 18:38:24 dist-test-slave-1xrh krb5kdc[9325](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041504, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 18:38:24.395898 9338 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260806 18:38:24.396744 9338 webserver.cc:460] Webserver started at http://127.0.0.1:39577/ using document root <none> and password file <none>
I20260806 18:38:24.397284 9338 fs_manager.cc:359] Metadata directory not provided
I20260806 18:38:24.397369 9338 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 18:38:24.397583 9338 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 18:38:24.399531 9338 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786041246094603-20232-0/minicluster-data/master-0/data/instance:
uuid: "2915d2fc7c3149c1a2ecda45e8dba5b7"
format_stamp: "Formatted at 2026-08-06 18:38:24 on dist-test-slave-1xrh"
server_key: "bf7ab763eb1c49e64cc7bc2b8bf70dc2"
server_key_iv: "14d575367048c9cff6fb46adb3b6a8e6"
server_key_version: "encryptionkey@0"
I20260806 18:38:24.400063 9338 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786041246094603-20232-0/minicluster-data/master-0/wal/instance:
uuid: "2915d2fc7c3149c1a2ecda45e8dba5b7"
format_stamp: "Formatted at 2026-08-06 18:38:24 on dist-test-slave-1xrh"
server_key: "bf7ab763eb1c49e64cc7bc2b8bf70dc2"
server_key_iv: "14d575367048c9cff6fb46adb3b6a8e6"
server_key_version: "encryptionkey@0"
I20260806 18:38:24.403600 9338 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.001s
I20260806 18:38:24.405838 9354 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 18:38:24.406961 9338 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260806 18:38:24.407099 9338 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786041246094603-20232-0/minicluster-data/master-0/data,/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786041246094603-20232-0/minicluster-data/master-0/wal
uuid: "2915d2fc7c3149c1a2ecda45e8dba5b7"
format_stamp: "Formatted at 2026-08-06 18:38:24 on dist-test-slave-1xrh"
server_key: "bf7ab763eb1c49e64cc7bc2b8bf70dc2"
server_key_iv: "14d575367048c9cff6fb46adb3b6a8e6"
server_key_version: "encryptionkey@0"
I20260806 18:38:24.407258 9338 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786041246094603-20232-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786041246094603-20232-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786041246094603-20232-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 18:38:24.418738 9338 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 18:38:24.421337 9338 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 18:38:24.421527 9338 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 18:38:24.428871 9338 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:46723
I20260806 18:38:24.428900 9410 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:46723 every 8 connection(s)
I20260806 18:38:24.429729 9338 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786041246094603-20232-0/minicluster-data/master-0/data/info.pb
I20260806 18:38:24.432340 9411 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 18:38:24.433221 20232 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJW69tn/build/debug/bin/kudu as pid 9338
I20260806 18:38:24.433343 20232 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786041246094603-20232-0/minicluster-data/master-0/wal/instance
I20260806 18:38:24.433635 20232 external_mini_cluster.cc:1428] Setting key 95509d49c13663cc66ed9601a1dd27e8
I20260806 18:38:24.438094 9411 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2915d2fc7c3149c1a2ecda45e8dba5b7: Bootstrap starting.
I20260806 18:38:24.440337 9411 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 2915d2fc7c3149c1a2ecda45e8dba5b7: Neither blocks nor log segments found. Creating new log.
I20260806 18:38:24.441042 9411 log.cc:824] T 00000000000000000000000000000000 P 2915d2fc7c3149c1a2ecda45e8dba5b7: Log is configured to *not* fsync() on all Append() calls
Aug 06 18:38:24 dist-test-slave-1xrh krb5kdc[9325](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041504, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260806 18:38:24.443296 9411 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2915d2fc7c3149c1a2ecda45e8dba5b7: No bootstrap required, opened a new log
I20260806 18:38:24.447113 9411 raft_consensus.cc:348] T 00000000000000000000000000000000 P 2915d2fc7c3149c1a2ecda45e8dba5b7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2915d2fc7c3149c1a2ecda45e8dba5b7" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46723 } }
I20260806 18:38:24.447384 9411 raft_consensus.cc:374] T 00000000000000000000000000000000 P 2915d2fc7c3149c1a2ecda45e8dba5b7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 18:38:24.447465 9411 raft_consensus.cc:729] T 00000000000000000000000000000000 P 2915d2fc7c3149c1a2ecda45e8dba5b7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2915d2fc7c3149c1a2ecda45e8dba5b7, State: Initialized, Role: FOLLOWER
I20260806 18:38:24.447670 9414 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:38:24.434991 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43111 (local address 127.0.0.1:46723)
0806 18:38:24.435450 (+ 459us) server_negotiation.cc:206] Beginning negotiation
0806 18:38:24.435464 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0806 18:38:24.435498 (+ 34us) server_negotiation.cc:407] Connection header received
0806 18:38:24.436093 (+ 595us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:38:24.436108 (+ 15us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:38:24.436413 (+ 305us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:38:24.436739 (+ 326us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:38:24.437578 (+ 839us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:38:24.438544 (+ 966us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:38:24.439435 (+ 891us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:38:24.439907 (+ 472us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:38:24.440086 (+ 179us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:38:24.442986 (+ 2900us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:38:24.442991 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:38:24.443003 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 18:38:24.443028 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:38:24.444746 (+ 1718us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:38:24.445311 (+ 565us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:38:24.445315 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:38:24.445319 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:38:24.445436 (+ 117us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:38:24.445730 (+ 294us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:38:24.445732 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:38:24.445733 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:38:24.446088 (+ 355us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:38:24.446245 (+ 157us) server_negotiation.cc:1035] Waiting for connection context
0806 18:38:24.446538 (+ 293us) server_negotiation.cc:299] Negotiation successful
0806 18:38:24.446844 (+ 306us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":320,"thread_start_us":164,"threads_started":1}
I20260806 18:38:24.447964 9411 consensus_queue.cc:260] T 00000000000000000000000000000000 P 2915d2fc7c3149c1a2ecda45e8dba5b7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2915d2fc7c3149c1a2ecda45e8dba5b7" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46723 } }
I20260806 18:38:24.448128 9411 raft_consensus.cc:388] T 00000000000000000000000000000000 P 2915d2fc7c3149c1a2ecda45e8dba5b7 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 18:38:24.448192 9411 raft_consensus.cc:482] T 00000000000000000000000000000000 P 2915d2fc7c3149c1a2ecda45e8dba5b7 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 18:38:24.448285 9411 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 2915d2fc7c3149c1a2ecda45e8dba5b7 [term 0 FOLLOWER]: Advancing to term 1
I20260806 18:38:24.449191 9411 raft_consensus.cc:504] T 00000000000000000000000000000000 P 2915d2fc7c3149c1a2ecda45e8dba5b7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2915d2fc7c3149c1a2ecda45e8dba5b7" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46723 } }
I20260806 18:38:24.449546 9411 leader_election.cc:302] T 00000000000000000000000000000000 P 2915d2fc7c3149c1a2ecda45e8dba5b7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 2915d2fc7c3149c1a2ecda45e8dba5b7; no voters:
I20260806 18:38:24.449875 9411 leader_election.cc:288] T 00000000000000000000000000000000 P 2915d2fc7c3149c1a2ecda45e8dba5b7 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 18:38:24.449954 9416 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 2915d2fc7c3149c1a2ecda45e8dba5b7 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 18:38:24.450141 9416 raft_consensus.cc:686] T 00000000000000000000000000000000 P 2915d2fc7c3149c1a2ecda45e8dba5b7 [term 1 LEADER]: Becoming Leader. State: Replica: 2915d2fc7c3149c1a2ecda45e8dba5b7, State: Running, Role: LEADER
I20260806 18:38:24.450412 9416 consensus_queue.cc:237] T 00000000000000000000000000000000 P 2915d2fc7c3149c1a2ecda45e8dba5b7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2915d2fc7c3149c1a2ecda45e8dba5b7" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46723 } }
I20260806 18:38:24.450481 20232 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260806 18:38:24.451094 9411 sys_catalog.cc:564] T 00000000000000000000000000000000 P 2915d2fc7c3149c1a2ecda45e8dba5b7 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 18:38:24.452248 9419 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2915d2fc7c3149c1a2ecda45e8dba5b7 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 2915d2fc7c3149c1a2ecda45e8dba5b7. Latest consensus state: current_term: 1 leader_uuid: "2915d2fc7c3149c1a2ecda45e8dba5b7" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2915d2fc7c3149c1a2ecda45e8dba5b7" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46723 } } }
I20260806 18:38:24.452394 9419 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2915d2fc7c3149c1a2ecda45e8dba5b7 [sys.catalog]: This master's current role is: LEADER
I20260806 18:38:24.452807 9431 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 18:38:24.452980 9417 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2915d2fc7c3149c1a2ecda45e8dba5b7 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "2915d2fc7c3149c1a2ecda45e8dba5b7" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2915d2fc7c3149c1a2ecda45e8dba5b7" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46723 } } }
I20260806 18:38:24.453080 9417 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2915d2fc7c3149c1a2ecda45e8dba5b7 [sys.catalog]: This master's current role is: LEADER
I20260806 18:38:24.456069 9431 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 18:38:24.462190 9414 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:38:24.452743 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60837 (local address 127.0.0.1:46723)
0806 18:38:24.452850 (+ 107us) server_negotiation.cc:206] Beginning negotiation
0806 18:38:24.452854 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 18:38:24.453217 (+ 363us) server_negotiation.cc:407] Connection header received
0806 18:38:24.453387 (+ 170us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:38:24.453391 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:38:24.453449 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:38:24.453502 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:38:24.454401 (+ 899us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:38:24.455098 (+ 697us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:38:24.456299 (+ 1201us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:38:24.456715 (+ 416us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:38:24.456797 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:38:24.458646 (+ 1849us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:38:24.458650 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:38:24.458652 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 18:38:24.458676 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:38:24.460342 (+ 1666us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:38:24.460853 (+ 511us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:38:24.460857 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:38:24.460859 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:38:24.460957 (+ 98us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:38:24.461281 (+ 324us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:38:24.461285 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:38:24.461286 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:38:24.461532 (+ 246us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:38:24.461673 (+ 141us) server_negotiation.cc:1035] Waiting for connection context
0806 18:38:24.461935 (+ 262us) server_negotiation.cc:299] Negotiation successful
0806 18:38:24.462048 (+ 113us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260806 18:38:24.462392 9431 catalog_manager.cc:1347] Generated new cluster ID: eb1f7c8cedf3486f80d1f853e74a4f22
I20260806 18:38:24.462442 9431 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 18:38:24.491711 9431 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 18:38:24.492671 9431 catalog_manager.cc:1504] Loading token signing keys...
I20260806 18:38:24.499457 9431 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 2915d2fc7c3149c1a2ecda45e8dba5b7: Generated new TSK 0
I20260806 18:38:24.500093 9431 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 18:38:24.503925 20232 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJW69tn/build/debug/bin/kudu with pid 9338
2026-08-06T18:38:24Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/3 (1431 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/4
Loading random data
Initializing database '/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786041246094603-20232-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 18:38:24 dist-test-slave-1xrh krb5kdc[9444](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 18:38:24 dist-test-slave-1xrh krb5kdc[9444](info): set up 2 sockets
Aug 06 18:38:24 dist-test-slave-1xrh krb5kdc[9444](info): commencing operation
krb5kdc: starting...
W20260806 18:38:25.552721 20232 mini_kdc.cc:121] Time spent starting KDC: real 1.027s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786041246094603-20232-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786041246094603-20232-0/krb5kdc/test-user.keytab.
Aug 06 18:38:25 dist-test-slave-1xrh krb5kdc[9444](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041505, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T18:38:25Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T18:38:25Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786041246094603-20232-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786041246094603-20232-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260806 18:38:25.670328 20232 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJW69tn/build/debug/bin/kudu
/tmp/dist-test-taskJW69tn/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786041246094603-20232-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786041246094603-20232-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786041246094603-20232-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786041246094603-20232-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:37065
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786041246094603-20232-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:44141
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:37065
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786041246094603-20232-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786041246094603-20232-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786041246094603-20232-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 18:38:25.779088 9458 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 18:38:25.779464 9458 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 18:38:25.779515 9458 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 18:38:25.784247 9458 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 18:38:25.784307 9458 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 18:38:25.784327 9458 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 18:38:25.784344 9458 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 18:38:25.784497 9458 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 18:38:25.790164 9458 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:44141
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786041246094603-20232-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786041246094603-20232-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:37065
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786041246094603-20232-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:37065
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786041246094603-20232-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9458
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786041246094603-20232-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 17:59:13 UTC on ccycloud.quasar-wvsnzx.root.comops.site
build id 4571028
I20260806 18:38:25.791306 9458 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 18:38:25.792315 9458 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 18:38:25.801383 9466 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:38:25.801390 9464 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:38:25.801430 9463 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 18:38:25.801669 9458 server_base.cc:1034] running on GCE node
I20260806 18:38:25.802315 9458 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 18:38:25.803400 9458 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 18:38:25.804589 9458 hybrid_clock.cc:648] HybridClock initialized: now 1786041505804563 us; error 37 us; skew 500 ppm
Aug 06 18:38:25 dist-test-slave-1xrh krb5kdc[9444](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041505, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 18:38:25.807343 9458 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260806 18:38:25.808149 9458 webserver.cc:460] Webserver started at http://127.0.0.1:38119/ using document root <none> and password file <none>
I20260806 18:38:25.808684 9458 fs_manager.cc:359] Metadata directory not provided
I20260806 18:38:25.808773 9458 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 18:38:25.809005 9458 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 18:38:25.810936 9458 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786041246094603-20232-0/minicluster-data/master-0/data/instance:
uuid: "335ddd6fa9444ef890e4d3b9a9807614"
format_stamp: "Formatted at 2026-08-06 18:38:25 on dist-test-slave-1xrh"
server_key: "483e427350fbc176a2c98352c0dd6e6a"
server_key_iv: "d2439b7425a814113b433971c774aacb"
server_key_version: "encryptionkey@0"
I20260806 18:38:25.811542 9458 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786041246094603-20232-0/minicluster-data/master-0/wal/instance:
uuid: "335ddd6fa9444ef890e4d3b9a9807614"
format_stamp: "Formatted at 2026-08-06 18:38:25 on dist-test-slave-1xrh"
server_key: "483e427350fbc176a2c98352c0dd6e6a"
server_key_iv: "d2439b7425a814113b433971c774aacb"
server_key_version: "encryptionkey@0"
I20260806 18:38:25.814996 9458 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.000s
I20260806 18:38:25.817328 9474 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 18:38:25.818493 9458 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260806 18:38:25.818629 9458 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786041246094603-20232-0/minicluster-data/master-0/data,/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786041246094603-20232-0/minicluster-data/master-0/wal
uuid: "335ddd6fa9444ef890e4d3b9a9807614"
format_stamp: "Formatted at 2026-08-06 18:38:25 on dist-test-slave-1xrh"
server_key: "483e427350fbc176a2c98352c0dd6e6a"
server_key_iv: "d2439b7425a814113b433971c774aacb"
server_key_version: "encryptionkey@0"
I20260806 18:38:25.818735 9458 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786041246094603-20232-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786041246094603-20232-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786041246094603-20232-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 18:38:25.850630 9458 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 18:38:25.853914 9458 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 18:38:25.854127 9458 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 18:38:25.861407 9458 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:37065
I20260806 18:38:25.861436 9530 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:37065 every 8 connection(s)
I20260806 18:38:25.862336 9458 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786041246094603-20232-0/minicluster-data/master-0/data/info.pb
I20260806 18:38:25.865057 9531 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 18:38:25.866366 20232 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJW69tn/build/debug/bin/kudu as pid 9458
I20260806 18:38:25.866516 20232 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786041246094603-20232-0/minicluster-data/master-0/wal/instance
I20260806 18:38:25.866813 20232 external_mini_cluster.cc:1428] Setting key 621468597ad1eb5c88e3a978eaf74440
I20260806 18:38:25.870932 9531 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 335ddd6fa9444ef890e4d3b9a9807614: Bootstrap starting.
I20260806 18:38:25.873181 9531 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 335ddd6fa9444ef890e4d3b9a9807614: Neither blocks nor log segments found. Creating new log.
I20260806 18:38:25.874012 9531 log.cc:824] T 00000000000000000000000000000000 P 335ddd6fa9444ef890e4d3b9a9807614: Log is configured to *not* fsync() on all Append() calls
I20260806 18:38:25.875712 9534 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:38:25.868255 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49867 (local address 127.0.0.1:37065)
0806 18:38:25.868674 (+ 419us) server_negotiation.cc:206] Beginning negotiation
0806 18:38:25.868701 (+ 27us) server_negotiation.cc:399] Waiting for connection header
0806 18:38:25.868735 (+ 34us) server_negotiation.cc:407] Connection header received
0806 18:38:25.869353 (+ 618us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:38:25.869387 (+ 34us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:38:25.869704 (+ 317us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:38:25.870015 (+ 311us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:38:25.870436 (+ 421us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:38:25.871478 (+ 1042us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:38:25.872374 (+ 896us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:38:25.872904 (+ 530us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:38:25.873081 (+ 177us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:38:25.873981 (+ 900us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:38:25.873987 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:38:25.873998 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 18:38:25.874430 (+ 432us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:38:25.874483 (+ 53us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 18:38:25.874506 (+ 23us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:38:25.874614 (+ 108us) server_negotiation.cc:1035] Waiting for connection context
0806 18:38:25.874762 (+ 148us) server_negotiation.cc:299] Negotiation successful
0806 18:38:25.874960 (+ 198us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":277,"thread_start_us":132,"threads_started":1}
I20260806 18:38:25.876008 9531 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 335ddd6fa9444ef890e4d3b9a9807614: No bootstrap required, opened a new log
I20260806 18:38:25.878417 20232 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260806 18:38:25.879843 9531 raft_consensus.cc:348] T 00000000000000000000000000000000 P 335ddd6fa9444ef890e4d3b9a9807614 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "335ddd6fa9444ef890e4d3b9a9807614" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37065 } }
I20260806 18:38:25.880074 9531 raft_consensus.cc:374] T 00000000000000000000000000000000 P 335ddd6fa9444ef890e4d3b9a9807614 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 18:38:25.880147 9531 raft_consensus.cc:729] T 00000000000000000000000000000000 P 335ddd6fa9444ef890e4d3b9a9807614 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 335ddd6fa9444ef890e4d3b9a9807614, State: Initialized, Role: FOLLOWER
I20260806 18:38:25.880667 9531 consensus_queue.cc:260] T 00000000000000000000000000000000 P 335ddd6fa9444ef890e4d3b9a9807614 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "335ddd6fa9444ef890e4d3b9a9807614" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37065 } }
I20260806 18:38:25.880846 9531 raft_consensus.cc:388] T 00000000000000000000000000000000 P 335ddd6fa9444ef890e4d3b9a9807614 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 18:38:25.880913 9531 raft_consensus.cc:482] T 00000000000000000000000000000000 P 335ddd6fa9444ef890e4d3b9a9807614 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 18:38:25.881011 9531 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 335ddd6fa9444ef890e4d3b9a9807614 [term 0 FOLLOWER]: Advancing to term 1
I20260806 18:38:25.882099 9531 raft_consensus.cc:504] T 00000000000000000000000000000000 P 335ddd6fa9444ef890e4d3b9a9807614 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "335ddd6fa9444ef890e4d3b9a9807614" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37065 } }
I20260806 18:38:25.882450 9531 leader_election.cc:302] T 00000000000000000000000000000000 P 335ddd6fa9444ef890e4d3b9a9807614 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 335ddd6fa9444ef890e4d3b9a9807614; no voters:
I20260806 18:38:25.882740 9531 leader_election.cc:288] T 00000000000000000000000000000000 P 335ddd6fa9444ef890e4d3b9a9807614 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 18:38:25.882858 9541 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 335ddd6fa9444ef890e4d3b9a9807614 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 18:38:25.883095 9541 raft_consensus.cc:686] T 00000000000000000000000000000000 P 335ddd6fa9444ef890e4d3b9a9807614 [term 1 LEADER]: Becoming Leader. State: Replica: 335ddd6fa9444ef890e4d3b9a9807614, State: Running, Role: LEADER
I20260806 18:38:25.883492 9541 consensus_queue.cc:237] T 00000000000000000000000000000000 P 335ddd6fa9444ef890e4d3b9a9807614 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "335ddd6fa9444ef890e4d3b9a9807614" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37065 } }
I20260806 18:38:25.883929 9531 sys_catalog.cc:564] T 00000000000000000000000000000000 P 335ddd6fa9444ef890e4d3b9a9807614 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 18:38:25.884761 9541 sys_catalog.cc:455] T 00000000000000000000000000000000 P 335ddd6fa9444ef890e4d3b9a9807614 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 335ddd6fa9444ef890e4d3b9a9807614. Latest consensus state: current_term: 1 leader_uuid: "335ddd6fa9444ef890e4d3b9a9807614" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "335ddd6fa9444ef890e4d3b9a9807614" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37065 } } }
I20260806 18:38:25.884953 9541 sys_catalog.cc:458] T 00000000000000000000000000000000 P 335ddd6fa9444ef890e4d3b9a9807614 [sys.catalog]: This master's current role is: LEADER
I20260806 18:38:25.885201 9542 sys_catalog.cc:455] T 00000000000000000000000000000000 P 335ddd6fa9444ef890e4d3b9a9807614 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "335ddd6fa9444ef890e4d3b9a9807614" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "335ddd6fa9444ef890e4d3b9a9807614" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37065 } } }
I20260806 18:38:25.885425 9551 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 18:38:25.886008 9542 sys_catalog.cc:458] T 00000000000000000000000000000000 P 335ddd6fa9444ef890e4d3b9a9807614 [sys.catalog]: This master's current role is: LEADER
I20260806 18:38:25.885746 9534 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:38:25.880501 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47423 (local address 127.0.0.1:37065)
0806 18:38:25.880614 (+ 113us) server_negotiation.cc:206] Beginning negotiation
0806 18:38:25.880618 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 18:38:25.880807 (+ 189us) server_negotiation.cc:407] Connection header received
0806 18:38:25.880954 (+ 147us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:38:25.880958 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:38:25.881002 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:38:25.881084 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:38:25.881355 (+ 271us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:38:25.882108 (+ 753us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:38:25.883221 (+ 1113us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:38:25.883558 (+ 337us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:38:25.883636 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:38:25.884682 (+ 1046us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:38:25.884689 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:38:25.884696 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 18:38:25.884712 (+ 16us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:38:25.884744 (+ 32us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 18:38:25.884765 (+ 21us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:38:25.884846 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0806 18:38:25.885520 (+ 674us) server_negotiation.cc:299] Negotiation successful
0806 18:38:25.885580 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260806 18:38:25.888983 9551 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 18:38:25.894341 9551 catalog_manager.cc:1347] Generated new cluster ID: 0ef41435baec4f12ac152a57bee00dcb
I20260806 18:38:25.894400 9551 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 18:38:25.944768 9551 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 18:38:25.945783 9551 catalog_manager.cc:1504] Loading token signing keys...
I20260806 18:38:25.954100 9551 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 335ddd6fa9444ef890e4d3b9a9807614: Generated new TSK 0
I20260806 18:38:25.954764 9551 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 18:38:25.963307 20232 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJW69tn/build/debug/bin/kudu with pid 9458
2026-08-06T18:38:25Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/4 (1459 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/5
Loading random data
Initializing database '/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786041246094603-20232-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 18:38:26 dist-test-slave-1xrh krb5kdc[9564](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 18:38:26 dist-test-slave-1xrh krb5kdc[9564](info): set up 2 sockets
Aug 06 18:38:26 dist-test-slave-1xrh krb5kdc[9564](info): commencing operation
krb5kdc: starting...
W20260806 18:38:27.021308 20232 mini_kdc.cc:121] Time spent starting KDC: real 1.036s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786041246094603-20232-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786041246094603-20232-0/krb5kdc/test-user.keytab.
Aug 06 18:38:27 dist-test-slave-1xrh krb5kdc[9564](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041507, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T18:38:27Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T18:38:27Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786041246094603-20232-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786041246094603-20232-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260806 18:38:27.142053 20232 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJW69tn/build/debug/bin/kudu
/tmp/dist-test-taskJW69tn/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786041246094603-20232-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786041246094603-20232-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786041246094603-20232-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786041246094603-20232-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:44579
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786041246094603-20232-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:44197
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:44579
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786041246094603-20232-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786041246094603-20232-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786041246094603-20232-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 18:38:27.249926 9579 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 18:38:27.250151 9579 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 18:38:27.250190 9579 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 18:38:27.254063 9579 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 18:38:27.254112 9579 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 18:38:27.254130 9579 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 18:38:27.254145 9579 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 18:38:27.254159 9579 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 18:38:27.259008 9579 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:44197
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786041246094603-20232-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786041246094603-20232-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:44579
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786041246094603-20232-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:44579
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786041246094603-20232-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9579
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786041246094603-20232-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 17:59:13 UTC on ccycloud.quasar-wvsnzx.root.comops.site
build id 4571028
I20260806 18:38:27.259912 9579 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 18:38:27.260804 9579 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260806 18:38:27.269039 9579 server_base.cc:1034] running on GCE node
W20260806 18:38:27.268965 9585 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:38:27.269002 9584 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:38:27.268965 9587 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 18:38:27.269739 9579 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 18:38:27.270705 9579 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 18:38:27.271859 9579 hybrid_clock.cc:648] HybridClock initialized: now 1786041507271842 us; error 32 us; skew 500 ppm
Aug 06 18:38:27 dist-test-slave-1xrh krb5kdc[9564](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041507, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 18:38:27.274442 9579 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260806 18:38:27.275309 9579 webserver.cc:460] Webserver started at http://127.0.0.1:34143/ using document root <none> and password file <none>
I20260806 18:38:27.275797 9579 fs_manager.cc:359] Metadata directory not provided
I20260806 18:38:27.275854 9579 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 18:38:27.276013 9579 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 18:38:27.278041 9579 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786041246094603-20232-0/minicluster-data/master-0/data/instance:
uuid: "e4dfd91933124b51aab86aaa9363c0f2"
format_stamp: "Formatted at 2026-08-06 18:38:27 on dist-test-slave-1xrh"
server_key: "2e2e4544411354440eb212395262270b"
server_key_iv: "e645e35c80093cf529f701d13df2bf7d"
server_key_version: "encryptionkey@0"
I20260806 18:38:27.278533 9579 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786041246094603-20232-0/minicluster-data/master-0/wal/instance:
uuid: "e4dfd91933124b51aab86aaa9363c0f2"
format_stamp: "Formatted at 2026-08-06 18:38:27 on dist-test-slave-1xrh"
server_key: "2e2e4544411354440eb212395262270b"
server_key_iv: "e645e35c80093cf529f701d13df2bf7d"
server_key_version: "encryptionkey@0"
I20260806 18:38:27.281937 9579 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.003s
I20260806 18:38:27.284225 9594 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 18:38:27.285392 9579 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260806 18:38:27.285526 9579 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786041246094603-20232-0/minicluster-data/master-0/data,/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786041246094603-20232-0/minicluster-data/master-0/wal
uuid: "e4dfd91933124b51aab86aaa9363c0f2"
format_stamp: "Formatted at 2026-08-06 18:38:27 on dist-test-slave-1xrh"
server_key: "2e2e4544411354440eb212395262270b"
server_key_iv: "e645e35c80093cf529f701d13df2bf7d"
server_key_version: "encryptionkey@0"
I20260806 18:38:27.285624 9579 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786041246094603-20232-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786041246094603-20232-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786041246094603-20232-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 18:38:27.290187 9579 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 18:38:27.292971 9579 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 18:38:27.293162 9579 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 18:38:27.300096 9579 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:44579
I20260806 18:38:27.300150 9649 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:44579 every 8 connection(s)
I20260806 18:38:27.300984 9579 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786041246094603-20232-0/minicluster-data/master-0/data/info.pb
I20260806 18:38:27.303664 9650 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 18:38:27.307690 20232 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJW69tn/build/debug/bin/kudu as pid 9579
I20260806 18:38:27.307842 20232 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786041246094603-20232-0/minicluster-data/master-0/wal/instance
I20260806 18:38:27.308115 20232 external_mini_cluster.cc:1428] Setting key 04046f6e6b397e6e2498381378480d21
I20260806 18:38:27.309346 9650 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e4dfd91933124b51aab86aaa9363c0f2: Bootstrap starting.
I20260806 18:38:27.312580 9650 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P e4dfd91933124b51aab86aaa9363c0f2: Neither blocks nor log segments found. Creating new log.
I20260806 18:38:27.313582 9650 log.cc:824] T 00000000000000000000000000000000 P e4dfd91933124b51aab86aaa9363c0f2: Log is configured to *not* fsync() on all Append() calls
I20260806 18:38:27.314786 9653 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:38:27.309409 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33875 (local address 127.0.0.1:44579)
0806 18:38:27.310111 (+ 702us) server_negotiation.cc:206] Beginning negotiation
0806 18:38:27.310132 (+ 21us) server_negotiation.cc:399] Waiting for connection header
0806 18:38:27.310180 (+ 48us) server_negotiation.cc:407] Connection header received
0806 18:38:27.311261 (+ 1081us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:38:27.311288 (+ 27us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:38:27.311691 (+ 403us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:38:27.312105 (+ 414us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:38:27.313178 (+ 1073us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:38:27.313187 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:38:27.313207 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 18:38:27.313223 (+ 16us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:38:27.313306 (+ 83us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 18:38:27.313336 (+ 30us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:38:27.313455 (+ 119us) server_negotiation.cc:1035] Waiting for connection context
0806 18:38:27.313666 (+ 211us) server_negotiation.cc:299] Negotiation successful
0806 18:38:27.313878 (+ 212us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":503,"thread_start_us":143,"threads_started":1}
I20260806 18:38:27.315965 9650 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e4dfd91933124b51aab86aaa9363c0f2: No bootstrap required, opened a new log
I20260806 18:38:27.317852 20232 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260806 18:38:27.320546 9650 raft_consensus.cc:348] T 00000000000000000000000000000000 P e4dfd91933124b51aab86aaa9363c0f2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e4dfd91933124b51aab86aaa9363c0f2" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44579 } }
I20260806 18:38:27.320827 9650 raft_consensus.cc:374] T 00000000000000000000000000000000 P e4dfd91933124b51aab86aaa9363c0f2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 18:38:27.320894 9650 raft_consensus.cc:729] T 00000000000000000000000000000000 P e4dfd91933124b51aab86aaa9363c0f2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e4dfd91933124b51aab86aaa9363c0f2, State: Initialized, Role: FOLLOWER
I20260806 18:38:27.321313 9650 consensus_queue.cc:260] T 00000000000000000000000000000000 P e4dfd91933124b51aab86aaa9363c0f2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e4dfd91933124b51aab86aaa9363c0f2" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44579 } }
I20260806 18:38:27.321511 9650 raft_consensus.cc:388] T 00000000000000000000000000000000 P e4dfd91933124b51aab86aaa9363c0f2 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 18:38:27.321585 9650 raft_consensus.cc:482] T 00000000000000000000000000000000 P e4dfd91933124b51aab86aaa9363c0f2 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 18:38:27.321686 9650 raft_consensus.cc:3037] T 00000000000000000000000000000000 P e4dfd91933124b51aab86aaa9363c0f2 [term 0 FOLLOWER]: Advancing to term 1
I20260806 18:38:27.321789 9653 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:38:27.320197 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48655 (local address 127.0.0.1:44579)
0806 18:38:27.320323 (+ 126us) server_negotiation.cc:206] Beginning negotiation
0806 18:38:27.320327 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 18:38:27.320459 (+ 132us) server_negotiation.cc:407] Connection header received
0806 18:38:27.320540 (+ 81us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:38:27.320543 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:38:27.320578 (+ 35us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:38:27.320669 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:38:27.321320 (+ 651us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:38:27.321326 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:38:27.321332 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 18:38:27.321344 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:38:27.321390 (+ 46us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 18:38:27.321403 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:38:27.321495 (+ 92us) server_negotiation.cc:1035] Waiting for connection context
0806 18:38:27.321605 (+ 110us) server_negotiation.cc:299] Negotiation successful
0806 18:38:27.321644 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260806 18:38:27.322732 9650 raft_consensus.cc:504] T 00000000000000000000000000000000 P e4dfd91933124b51aab86aaa9363c0f2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e4dfd91933124b51aab86aaa9363c0f2" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44579 } }
I20260806 18:38:27.323030 9650 leader_election.cc:302] T 00000000000000000000000000000000 P e4dfd91933124b51aab86aaa9363c0f2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: e4dfd91933124b51aab86aaa9363c0f2; no voters:
I20260806 18:38:27.323333 9650 leader_election.cc:288] T 00000000000000000000000000000000 P e4dfd91933124b51aab86aaa9363c0f2 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 18:38:27.323454 9661 raft_consensus.cc:2781] T 00000000000000000000000000000000 P e4dfd91933124b51aab86aaa9363c0f2 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 18:38:27.323709 9661 raft_consensus.cc:686] T 00000000000000000000000000000000 P e4dfd91933124b51aab86aaa9363c0f2 [term 1 LEADER]: Becoming Leader. State: Replica: e4dfd91933124b51aab86aaa9363c0f2, State: Running, Role: LEADER
I20260806 18:38:27.324056 9661 consensus_queue.cc:237] T 00000000000000000000000000000000 P e4dfd91933124b51aab86aaa9363c0f2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e4dfd91933124b51aab86aaa9363c0f2" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44579 } }
I20260806 18:38:27.324494 9650 sys_catalog.cc:564] T 00000000000000000000000000000000 P e4dfd91933124b51aab86aaa9363c0f2 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 18:38:27.325706 9663 sys_catalog.cc:455] T 00000000000000000000000000000000 P e4dfd91933124b51aab86aaa9363c0f2 [sys.catalog]: SysCatalogTable state changed. Reason: New leader e4dfd91933124b51aab86aaa9363c0f2. Latest consensus state: current_term: 1 leader_uuid: "e4dfd91933124b51aab86aaa9363c0f2" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e4dfd91933124b51aab86aaa9363c0f2" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44579 } } }
I20260806 18:38:27.325757 9662 sys_catalog.cc:455] T 00000000000000000000000000000000 P e4dfd91933124b51aab86aaa9363c0f2 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "e4dfd91933124b51aab86aaa9363c0f2" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e4dfd91933124b51aab86aaa9363c0f2" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44579 } } }
I20260806 18:38:27.325827 9663 sys_catalog.cc:458] T 00000000000000000000000000000000 P e4dfd91933124b51aab86aaa9363c0f2 [sys.catalog]: This master's current role is: LEADER
I20260806 18:38:27.325860 9662 sys_catalog.cc:458] T 00000000000000000000000000000000 P e4dfd91933124b51aab86aaa9363c0f2 [sys.catalog]: This master's current role is: LEADER
I20260806 18:38:27.326207 9670 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 18:38:27.329210 9670 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 18:38:27.334165 9670 catalog_manager.cc:1347] Generated new cluster ID: 8d3d90e58ba14ec3897a1d7b7f514156
I20260806 18:38:27.334226 9670 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 18:38:27.347052 9670 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 18:38:27.347380 9670 catalog_manager.cc:1504] Loading token signing keys...
I20260806 18:38:27.356734 9670 catalog_manager.cc:5868] T 00000000000000000000000000000000 P e4dfd91933124b51aab86aaa9363c0f2: Generated new TSK 0
I20260806 18:38:27.357451 9670 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 18:38:27.366006 20232 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJW69tn/build/debug/bin/kudu with pid 9579
2026-08-06T18:38:27Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/5 (1401 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/6
Loading random data
Initializing database '/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786041246094603-20232-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 18:38:27 dist-test-slave-1xrh krb5kdc[9683](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 18:38:27 dist-test-slave-1xrh krb5kdc[9683](info): set up 2 sockets
Aug 06 18:38:27 dist-test-slave-1xrh krb5kdc[9683](info): commencing operation
krb5kdc: starting...
W20260806 18:38:28.412818 20232 mini_kdc.cc:121] Time spent starting KDC: real 1.025s user 0.004s sys 0.002s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786041246094603-20232-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786041246094603-20232-0/krb5kdc/test-user.keytab.
Aug 06 18:38:28 dist-test-slave-1xrh krb5kdc[9683](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041508, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T18:38:28Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T18:38:28Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786041246094603-20232-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786041246094603-20232-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260806 18:38:28.532445 20232 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJW69tn/build/debug/bin/kudu
/tmp/dist-test-taskJW69tn/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786041246094603-20232-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786041246094603-20232-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786041246094603-20232-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786041246094603-20232-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:33521
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786041246094603-20232-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:36655
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:33521
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786041246094603-20232-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786041246094603-20232-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786041246094603-20232-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 18:38:28.639533 9698 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 18:38:28.639755 9698 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 18:38:28.639792 9698 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 18:38:28.643528 9698 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 18:38:28.643576 9698 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 18:38:28.643595 9698 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 18:38:28.643611 9698 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 18:38:28.643626 9698 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 18:38:28.648599 9698 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:36655
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786041246094603-20232-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786041246094603-20232-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:33521
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786041246094603-20232-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:33521
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786041246094603-20232-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9698
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786041246094603-20232-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 17:59:13 UTC on ccycloud.quasar-wvsnzx.root.comops.site
build id 4571028
I20260806 18:38:28.649462 9698 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 18:38:28.650347 9698 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 18:38:28.658528 9703 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:38:28.658519 9707 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:38:28.658581 9704 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 18:38:28.658740 9698 server_base.cc:1034] running on GCE node
I20260806 18:38:28.659380 9698 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 18:38:28.660425 9698 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 18:38:28.661588 9698 hybrid_clock.cc:648] HybridClock initialized: now 1786041508661570 us; error 48 us; skew 500 ppm
Aug 06 18:38:28 dist-test-slave-1xrh krb5kdc[9683](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041508, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 18:38:28.664256 9698 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260806 18:38:28.665091 9698 webserver.cc:460] Webserver started at http://127.0.0.1:38065/ using document root <none> and password file <none>
I20260806 18:38:28.665618 9698 fs_manager.cc:359] Metadata directory not provided
I20260806 18:38:28.665707 9698 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 18:38:28.665937 9698 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 18:38:28.667945 9698 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786041246094603-20232-0/minicluster-data/master-0/data/instance:
uuid: "6d8a4875b93a44ddb2e4af48a12d2116"
format_stamp: "Formatted at 2026-08-06 18:38:28 on dist-test-slave-1xrh"
server_key: "34beae3406b5297071fed8589a920f9f"
server_key_iv: "10399732ff0ffa83877f65a44495abeb"
server_key_version: "encryptionkey@0"
I20260806 18:38:28.668488 9698 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786041246094603-20232-0/minicluster-data/master-0/wal/instance:
uuid: "6d8a4875b93a44ddb2e4af48a12d2116"
format_stamp: "Formatted at 2026-08-06 18:38:28 on dist-test-slave-1xrh"
server_key: "34beae3406b5297071fed8589a920f9f"
server_key_iv: "10399732ff0ffa83877f65a44495abeb"
server_key_version: "encryptionkey@0"
I20260806 18:38:28.671932 9698 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.001s
I20260806 18:38:28.674081 9715 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 18:38:28.675241 9698 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.001s
I20260806 18:38:28.675379 9698 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786041246094603-20232-0/minicluster-data/master-0/data,/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786041246094603-20232-0/minicluster-data/master-0/wal
uuid: "6d8a4875b93a44ddb2e4af48a12d2116"
format_stamp: "Formatted at 2026-08-06 18:38:28 on dist-test-slave-1xrh"
server_key: "34beae3406b5297071fed8589a920f9f"
server_key_iv: "10399732ff0ffa83877f65a44495abeb"
server_key_version: "encryptionkey@0"
I20260806 18:38:28.675493 9698 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786041246094603-20232-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786041246094603-20232-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786041246094603-20232-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 18:38:28.698120 9698 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 18:38:28.700891 9698 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 18:38:28.701094 9698 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 18:38:28.708189 9698 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:33521
I20260806 18:38:28.708298 9768 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:33521 every 8 connection(s)
I20260806 18:38:28.709110 9698 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786041246094603-20232-0/minicluster-data/master-0/data/info.pb
I20260806 18:38:28.711683 9769 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 18:38:28.716732 9769 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6d8a4875b93a44ddb2e4af48a12d2116: Bootstrap starting.
I20260806 18:38:28.718611 20232 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJW69tn/build/debug/bin/kudu as pid 9698
I20260806 18:38:28.718756 20232 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786041246094603-20232-0/minicluster-data/master-0/wal/instance
I20260806 18:38:28.719058 20232 external_mini_cluster.cc:1428] Setting key 1e94841e2c9f035a5bd4f272b0b825b5
I20260806 18:38:28.719270 9769 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 6d8a4875b93a44ddb2e4af48a12d2116: Neither blocks nor log segments found. Creating new log.
I20260806 18:38:28.720063 9769 log.cc:824] T 00000000000000000000000000000000 P 6d8a4875b93a44ddb2e4af48a12d2116: Log is configured to *not* fsync() on all Append() calls
I20260806 18:38:28.721874 9769 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6d8a4875b93a44ddb2e4af48a12d2116: No bootstrap required, opened a new log
I20260806 18:38:28.725255 9769 raft_consensus.cc:348] T 00000000000000000000000000000000 P 6d8a4875b93a44ddb2e4af48a12d2116 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6d8a4875b93a44ddb2e4af48a12d2116" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33521 } }
I20260806 18:38:28.725472 9769 raft_consensus.cc:374] T 00000000000000000000000000000000 P 6d8a4875b93a44ddb2e4af48a12d2116 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 18:38:28.725538 9769 raft_consensus.cc:729] T 00000000000000000000000000000000 P 6d8a4875b93a44ddb2e4af48a12d2116 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6d8a4875b93a44ddb2e4af48a12d2116, State: Initialized, Role: FOLLOWER
I20260806 18:38:28.726001 9769 consensus_queue.cc:260] T 00000000000000000000000000000000 P 6d8a4875b93a44ddb2e4af48a12d2116 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6d8a4875b93a44ddb2e4af48a12d2116" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33521 } }
I20260806 18:38:28.726145 9769 raft_consensus.cc:388] T 00000000000000000000000000000000 P 6d8a4875b93a44ddb2e4af48a12d2116 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 18:38:28.726227 9769 raft_consensus.cc:482] T 00000000000000000000000000000000 P 6d8a4875b93a44ddb2e4af48a12d2116 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 18:38:28.726342 9769 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 6d8a4875b93a44ddb2e4af48a12d2116 [term 0 FOLLOWER]: Advancing to term 1
I20260806 18:38:28.727427 9769 raft_consensus.cc:504] T 00000000000000000000000000000000 P 6d8a4875b93a44ddb2e4af48a12d2116 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6d8a4875b93a44ddb2e4af48a12d2116" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33521 } }
Aug 06 18:38:28 dist-test-slave-1xrh krb5kdc[9683](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041508, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260806 18:38:28.727802 9769 leader_election.cc:302] T 00000000000000000000000000000000 P 6d8a4875b93a44ddb2e4af48a12d2116 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 6d8a4875b93a44ddb2e4af48a12d2116; no voters:
I20260806 18:38:28.728129 9769 leader_election.cc:288] T 00000000000000000000000000000000 P 6d8a4875b93a44ddb2e4af48a12d2116 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 18:38:28.728226 9774 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 6d8a4875b93a44ddb2e4af48a12d2116 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 18:38:28.728456 9774 raft_consensus.cc:686] T 00000000000000000000000000000000 P 6d8a4875b93a44ddb2e4af48a12d2116 [term 1 LEADER]: Becoming Leader. State: Replica: 6d8a4875b93a44ddb2e4af48a12d2116, State: Running, Role: LEADER
I20260806 18:38:28.728780 9774 consensus_queue.cc:237] T 00000000000000000000000000000000 P 6d8a4875b93a44ddb2e4af48a12d2116 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6d8a4875b93a44ddb2e4af48a12d2116" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33521 } }
I20260806 18:38:28.729350 9769 sys_catalog.cc:564] T 00000000000000000000000000000000 P 6d8a4875b93a44ddb2e4af48a12d2116 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 18:38:28.731016 9775 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6d8a4875b93a44ddb2e4af48a12d2116 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "6d8a4875b93a44ddb2e4af48a12d2116" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6d8a4875b93a44ddb2e4af48a12d2116" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33521 } } }
I20260806 18:38:28.731158 9775 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6d8a4875b93a44ddb2e4af48a12d2116 [sys.catalog]: This master's current role is: LEADER
I20260806 18:38:28.731192 9776 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6d8a4875b93a44ddb2e4af48a12d2116 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 6d8a4875b93a44ddb2e4af48a12d2116. Latest consensus state: current_term: 1 leader_uuid: "6d8a4875b93a44ddb2e4af48a12d2116" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6d8a4875b93a44ddb2e4af48a12d2116" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33521 } } }
I20260806 18:38:28.731314 9776 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6d8a4875b93a44ddb2e4af48a12d2116 [sys.catalog]: This master's current role is: LEADER
I20260806 18:38:28.731606 9783 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 18:38:28.735303 9783 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 18:38:28.735359 9773 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:38:28.720204 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.186:40089 (local address 127.0.0.1:33521)
0806 18:38:28.720668 (+ 464us) server_negotiation.cc:206] Beginning negotiation
0806 18:38:28.720689 (+ 21us) server_negotiation.cc:399] Waiting for connection header
0806 18:38:28.720732 (+ 43us) server_negotiation.cc:407] Connection header received
0806 18:38:28.721383 (+ 651us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:38:28.721406 (+ 23us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:38:28.721714 (+ 308us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:38:28.722043 (+ 329us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:38:28.722871 (+ 828us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:38:28.723917 (+ 1046us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:38:28.724795 (+ 878us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:38:28.725252 (+ 457us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:38:28.725435 (+ 183us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:38:28.728619 (+ 3184us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:38:28.728626 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:38:28.728641 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 18:38:28.728670 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:38:28.731515 (+ 2845us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:38:28.732166 (+ 651us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:38:28.732173 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:38:28.732178 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:38:28.732339 (+ 161us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:38:28.732733 (+ 394us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:38:28.732737 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:38:28.732738 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:38:28.733232 (+ 494us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:38:28.733450 (+ 218us) server_negotiation.cc:1035] Waiting for connection context
0806 18:38:28.734227 (+ 777us) server_negotiation.cc:299] Negotiation successful
0806 18:38:28.734443 (+ 216us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":310,"thread_start_us":110,"threads_started":1}
I20260806 18:38:28.738291 20232 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260806 18:38:28.741516 9783 catalog_manager.cc:1347] Generated new cluster ID: c10d602d9b9948078b2ed6c9cb2c1ce3
I20260806 18:38:28.741627 9783 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 18:38:28.749205 9773 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:38:28.740381 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.186:48061 (local address 127.0.0.1:33521)
0806 18:38:28.740507 (+ 126us) server_negotiation.cc:206] Beginning negotiation
0806 18:38:28.740512 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 18:38:28.740711 (+ 199us) server_negotiation.cc:407] Connection header received
0806 18:38:28.740835 (+ 124us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:38:28.740838 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:38:28.740878 (+ 40us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:38:28.740969 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:38:28.741777 (+ 808us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:38:28.742592 (+ 815us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:38:28.743708 (+ 1116us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:38:28.744111 (+ 403us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:38:28.744222 (+ 111us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:38:28.745752 (+ 1530us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:38:28.745755 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:38:28.745758 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 18:38:28.745786 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:38:28.747413 (+ 1627us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:38:28.747933 (+ 520us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:38:28.747937 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:38:28.747938 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:38:28.748020 (+ 82us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:38:28.748357 (+ 337us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:38:28.748360 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:38:28.748362 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:38:28.748553 (+ 191us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:38:28.748698 (+ 145us) server_negotiation.cc:1035] Waiting for connection context
0806 18:38:28.748962 (+ 264us) server_negotiation.cc:299] Negotiation successful
0806 18:38:28.749058 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":27}
I20260806 18:38:28.753777 9783 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 18:38:28.754577 9783 catalog_manager.cc:1504] Loading token signing keys...
I20260806 18:38:28.762657 9783 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 6d8a4875b93a44ddb2e4af48a12d2116: Generated new TSK 0
I20260806 18:38:28.763345 9783 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 18:38:28.769539 20232 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJW69tn/build/debug/bin/kudu with pid 9698
W20260806 18:38:28.777480 9681 connection.cc:535] client connection to 127.0.0.1:33521 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:33521): Connection reset by peer (error 104)
2026-08-06T18:38:28Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/6 (1404 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/7
Loading random data
Initializing database '/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786041246094603-20232-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 18:38:28 dist-test-slave-1xrh krb5kdc[9801](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 18:38:28 dist-test-slave-1xrh krb5kdc[9801](info): set up 2 sockets
Aug 06 18:38:28 dist-test-slave-1xrh krb5kdc[9801](info): commencing operation
krb5kdc: starting...
W20260806 18:38:29.836758 20232 mini_kdc.cc:121] Time spent starting KDC: real 1.045s user 0.005s sys 0.001s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786041246094603-20232-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786041246094603-20232-0/krb5kdc/test-user.keytab.
Aug 06 18:38:29 dist-test-slave-1xrh krb5kdc[9801](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041509, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T18:38:29Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T18:38:29Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786041246094603-20232-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786041246094603-20232-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260806 18:38:29.958540 20232 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJW69tn/build/debug/bin/kudu
/tmp/dist-test-taskJW69tn/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786041246094603-20232-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786041246094603-20232-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786041246094603-20232-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786041246094603-20232-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:43301
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786041246094603-20232-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:37815
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:43301
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786041246094603-20232-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786041246094603-20232-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786041246094603-20232-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 18:38:30.069438 9815 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 18:38:30.069703 9815 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 18:38:30.069774 9815 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 18:38:30.073778 9815 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 18:38:30.073868 9815 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 18:38:30.073925 9815 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 18:38:30.073980 9815 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 18:38:30.074018 9815 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 18:38:30.079226 9815 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:37815
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786041246094603-20232-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786041246094603-20232-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:43301
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786041246094603-20232-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:43301
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786041246094603-20232-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9815
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786041246094603-20232-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 17:59:13 UTC on ccycloud.quasar-wvsnzx.root.comops.site
build id 4571028
I20260806 18:38:30.080260 9815 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 18:38:30.081236 9815 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 18:38:30.089656 9822 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:38:30.089689 9824 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:38:30.089675 9820 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 18:38:30.089811 9815 server_base.cc:1034] running on GCE node
I20260806 18:38:30.090374 9815 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 18:38:30.091490 9815 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 18:38:30.092671 9815 hybrid_clock.cc:648] HybridClock initialized: now 1786041510092656 us; error 31 us; skew 500 ppm
Aug 06 18:38:30 dist-test-slave-1xrh krb5kdc[9801](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041510, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 18:38:30.101346 9815 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260806 18:38:30.102185 9815 webserver.cc:460] Webserver started at http://127.0.0.1:33497/ using document root <none> and password file <none>
I20260806 18:38:30.102677 9815 fs_manager.cc:359] Metadata directory not provided
I20260806 18:38:30.102733 9815 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 18:38:30.102888 9815 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 18:38:30.104853 9815 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786041246094603-20232-0/minicluster-data/master-0/data/instance:
uuid: "db4cafb70f73486990a527033c8f5665"
format_stamp: "Formatted at 2026-08-06 18:38:30 on dist-test-slave-1xrh"
server_key: "c4546943de15242376171f9bcae78553"
server_key_iv: "9dd0f052e0d137b49f2ba80308579cc4"
server_key_version: "encryptionkey@0"
I20260806 18:38:30.105427 9815 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786041246094603-20232-0/minicluster-data/master-0/wal/instance:
uuid: "db4cafb70f73486990a527033c8f5665"
format_stamp: "Formatted at 2026-08-06 18:38:30 on dist-test-slave-1xrh"
server_key: "c4546943de15242376171f9bcae78553"
server_key_iv: "9dd0f052e0d137b49f2ba80308579cc4"
server_key_version: "encryptionkey@0"
I20260806 18:38:30.108986 9815 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.005s sys 0.000s
I20260806 18:38:30.111272 9832 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 18:38:30.112382 9815 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260806 18:38:30.112522 9815 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786041246094603-20232-0/minicluster-data/master-0/data,/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786041246094603-20232-0/minicluster-data/master-0/wal
uuid: "db4cafb70f73486990a527033c8f5665"
format_stamp: "Formatted at 2026-08-06 18:38:30 on dist-test-slave-1xrh"
server_key: "c4546943de15242376171f9bcae78553"
server_key_iv: "9dd0f052e0d137b49f2ba80308579cc4"
server_key_version: "encryptionkey@0"
I20260806 18:38:30.112622 9815 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786041246094603-20232-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786041246094603-20232-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786041246094603-20232-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 18:38:30.124640 9815 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 18:38:30.127779 9815 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 18:38:30.127977 9815 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 18:38:30.135058 9815 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:43301
I20260806 18:38:30.135085 9891 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:43301 every 8 connection(s)
I20260806 18:38:30.135963 9815 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786041246094603-20232-0/minicluster-data/master-0/data/info.pb
I20260806 18:38:30.138481 9892 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 18:38:30.143553 9892 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P db4cafb70f73486990a527033c8f5665: Bootstrap starting.
I20260806 18:38:30.144272 20232 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJW69tn/build/debug/bin/kudu as pid 9815
I20260806 18:38:30.144408 20232 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786041246094603-20232-0/minicluster-data/master-0/wal/instance
I20260806 18:38:30.144733 20232 external_mini_cluster.cc:1428] Setting key ee7e4369f43f0e095c3d35b1e0cdaf79
I20260806 18:38:30.146318 9892 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P db4cafb70f73486990a527033c8f5665: Neither blocks nor log segments found. Creating new log.
I20260806 18:38:30.147195 9892 log.cc:824] T 00000000000000000000000000000000 P db4cafb70f73486990a527033c8f5665: Log is configured to *not* fsync() on all Append() calls
I20260806 18:38:30.149261 9892 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P db4cafb70f73486990a527033c8f5665: No bootstrap required, opened a new log
I20260806 18:38:30.153475 9892 raft_consensus.cc:348] T 00000000000000000000000000000000 P db4cafb70f73486990a527033c8f5665 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "db4cafb70f73486990a527033c8f5665" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43301 } }
I20260806 18:38:30.153724 9892 raft_consensus.cc:374] T 00000000000000000000000000000000 P db4cafb70f73486990a527033c8f5665 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 18:38:30.153805 9892 raft_consensus.cc:729] T 00000000000000000000000000000000 P db4cafb70f73486990a527033c8f5665 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: db4cafb70f73486990a527033c8f5665, State: Initialized, Role: FOLLOWER
I20260806 18:38:30.154311 9892 consensus_queue.cc:260] T 00000000000000000000000000000000 P db4cafb70f73486990a527033c8f5665 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "db4cafb70f73486990a527033c8f5665" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43301 } }
I20260806 18:38:30.154469 9892 raft_consensus.cc:388] T 00000000000000000000000000000000 P db4cafb70f73486990a527033c8f5665 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 18:38:30.154469 9895 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:38:30.145926 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.186:56663 (local address 127.0.0.1:43301)
0806 18:38:30.146387 (+ 461us) server_negotiation.cc:206] Beginning negotiation
0806 18:38:30.146406 (+ 19us) server_negotiation.cc:399] Waiting for connection header
0806 18:38:30.146469 (+ 63us) server_negotiation.cc:407] Connection header received
0806 18:38:30.147328 (+ 859us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:38:30.147360 (+ 32us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:38:30.147741 (+ 381us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:38:30.148073 (+ 332us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:38:30.148488 (+ 415us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:38:30.149917 (+ 1429us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:38:30.150790 (+ 873us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:38:30.151509 (+ 719us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:38:30.151675 (+ 166us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:38:30.152465 (+ 790us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:38:30.152471 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:38:30.152486 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 18:38:30.152960 (+ 474us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:38:30.153025 (+ 65us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 18:38:30.153054 (+ 29us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:38:30.153182 (+ 128us) server_negotiation.cc:1035] Waiting for connection context
0806 18:38:30.153342 (+ 160us) server_negotiation.cc:299] Negotiation successful
0806 18:38:30.153476 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":302,"thread_start_us":121,"threads_started":1}
I20260806 18:38:30.154567 9892 raft_consensus.cc:482] T 00000000000000000000000000000000 P db4cafb70f73486990a527033c8f5665 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 18:38:30.154654 9892 raft_consensus.cc:3037] T 00000000000000000000000000000000 P db4cafb70f73486990a527033c8f5665 [term 0 FOLLOWER]: Advancing to term 1
I20260806 18:38:30.155632 9892 raft_consensus.cc:504] T 00000000000000000000000000000000 P db4cafb70f73486990a527033c8f5665 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "db4cafb70f73486990a527033c8f5665" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43301 } }
I20260806 18:38:30.155997 9892 leader_election.cc:302] T 00000000000000000000000000000000 P db4cafb70f73486990a527033c8f5665 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: db4cafb70f73486990a527033c8f5665; no voters:
I20260806 18:38:30.156303 9892 leader_election.cc:288] T 00000000000000000000000000000000 P db4cafb70f73486990a527033c8f5665 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 18:38:30.156430 9897 raft_consensus.cc:2781] T 00000000000000000000000000000000 P db4cafb70f73486990a527033c8f5665 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 18:38:30.156653 9897 raft_consensus.cc:686] T 00000000000000000000000000000000 P db4cafb70f73486990a527033c8f5665 [term 1 LEADER]: Becoming Leader. State: Replica: db4cafb70f73486990a527033c8f5665, State: Running, Role: LEADER
I20260806 18:38:30.157050 9897 consensus_queue.cc:237] T 00000000000000000000000000000000 P db4cafb70f73486990a527033c8f5665 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "db4cafb70f73486990a527033c8f5665" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43301 } }
I20260806 18:38:30.157621 9892 sys_catalog.cc:564] T 00000000000000000000000000000000 P db4cafb70f73486990a527033c8f5665 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 18:38:30.157696 20232 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260806 18:38:30.158756 9898 sys_catalog.cc:455] T 00000000000000000000000000000000 P db4cafb70f73486990a527033c8f5665 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "db4cafb70f73486990a527033c8f5665" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "db4cafb70f73486990a527033c8f5665" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43301 } } }
I20260806 18:38:30.158888 9898 sys_catalog.cc:458] T 00000000000000000000000000000000 P db4cafb70f73486990a527033c8f5665 [sys.catalog]: This master's current role is: LEADER
I20260806 18:38:30.159264 9899 sys_catalog.cc:455] T 00000000000000000000000000000000 P db4cafb70f73486990a527033c8f5665 [sys.catalog]: SysCatalogTable state changed. Reason: New leader db4cafb70f73486990a527033c8f5665. Latest consensus state: current_term: 1 leader_uuid: "db4cafb70f73486990a527033c8f5665" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "db4cafb70f73486990a527033c8f5665" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43301 } } }
I20260806 18:38:30.159368 9899 sys_catalog.cc:458] T 00000000000000000000000000000000 P db4cafb70f73486990a527033c8f5665 [sys.catalog]: This master's current role is: LEADER
I20260806 18:38:30.159662 9907 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 18:38:30.163235 9907 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 18:38:30.164285 9895 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:38:30.160174 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.186:34755 (local address 127.0.0.1:43301)
0806 18:38:30.160292 (+ 118us) server_negotiation.cc:206] Beginning negotiation
0806 18:38:30.160295 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 18:38:30.160755 (+ 460us) server_negotiation.cc:407] Connection header received
0806 18:38:30.160877 (+ 122us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:38:30.160879 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:38:30.160915 (+ 36us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:38:30.161011 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:38:30.161287 (+ 276us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:38:30.161802 (+ 515us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:38:30.162619 (+ 817us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:38:30.162853 (+ 234us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:38:30.162967 (+ 114us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:38:30.163843 (+ 876us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:38:30.163847 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:38:30.163850 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 18:38:30.163868 (+ 18us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:38:30.163890 (+ 22us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 18:38:30.163900 (+ 10us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:38:30.164000 (+ 100us) server_negotiation.cc:1035] Waiting for connection context
0806 18:38:30.164122 (+ 122us) server_negotiation.cc:299] Negotiation successful
0806 18:38:30.164153 (+ 31us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":28}
I20260806 18:38:30.168529 9907 catalog_manager.cc:1347] Generated new cluster ID: 5d0e3ca9952c4d1caaf338a15e43bc2b
I20260806 18:38:30.168602 9907 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 18:38:30.191022 9907 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 18:38:30.192214 9907 catalog_manager.cc:1504] Loading token signing keys...
I20260806 18:38:30.207355 9907 catalog_manager.cc:5868] T 00000000000000000000000000000000 P db4cafb70f73486990a527033c8f5665: Generated new TSK 0
I20260806 18:38:30.208119 9907 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 18:38:30.211380 20232 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJW69tn/build/debug/bin/kudu with pid 9815
W20260806 18:38:30.218997 9798 connection.cc:535] client connection to 127.0.0.1:43301 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:43301): Connection reset by peer (error 104)
2026-08-06T18:38:30Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/7 (1441 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/8
Loading random data
Initializing database '/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786041246094603-20232-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 18:38:30 dist-test-slave-1xrh krb5kdc[9923](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 18:38:30 dist-test-slave-1xrh krb5kdc[9923](info): set up 2 sockets
Aug 06 18:38:30 dist-test-slave-1xrh krb5kdc[9923](info): commencing operation
krb5kdc: starting...
W20260806 18:38:31.260704 20232 mini_kdc.cc:121] Time spent starting KDC: real 1.027s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786041246094603-20232-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786041246094603-20232-0/krb5kdc/test-user.keytab.
Aug 06 18:38:31 dist-test-slave-1xrh krb5kdc[9923](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041511, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T18:38:31Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T18:38:31Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786041246094603-20232-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786041246094603-20232-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260806 18:38:31.386053 20232 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJW69tn/build/debug/bin/kudu
/tmp/dist-test-taskJW69tn/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786041246094603-20232-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786041246094603-20232-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786041246094603-20232-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786041246094603-20232-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:34165
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786041246094603-20232-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:36255
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:34165
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786041246094603-20232-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786041246094603-20232-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786041246094603-20232-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 18:38:31.494062 9937 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 18:38:31.494318 9937 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 18:38:31.494359 9937 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 18:38:31.498238 9937 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 18:38:31.498289 9937 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 18:38:31.498308 9937 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 18:38:31.498324 9937 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 18:38:31.498363 9937 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 18:38:31.503522 9937 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:36255
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786041246094603-20232-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786041246094603-20232-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:34165
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786041246094603-20232-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:34165
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786041246094603-20232-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9937
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786041246094603-20232-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 17:59:13 UTC on ccycloud.quasar-wvsnzx.root.comops.site
build id 4571028
I20260806 18:38:31.504575 9937 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 18:38:31.505594 9937 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 18:38:31.513837 9943 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:38:31.513929 9945 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:38:31.513813 9942 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 18:38:31.514081 9937 server_base.cc:1034] running on GCE node
I20260806 18:38:31.514747 9937 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 18:38:31.515839 9937 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 18:38:31.517040 9937 hybrid_clock.cc:648] HybridClock initialized: now 1786041511517022 us; error 29 us; skew 500 ppm
Aug 06 18:38:31 dist-test-slave-1xrh krb5kdc[9923](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041511, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 18:38:31.519757 9937 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260806 18:38:31.520565 9937 webserver.cc:460] Webserver started at http://127.0.0.1:42893/ using document root <none> and password file <none>
I20260806 18:38:31.521131 9937 fs_manager.cc:359] Metadata directory not provided
I20260806 18:38:31.521234 9937 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 18:38:31.521456 9937 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 18:38:31.523449 9937 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786041246094603-20232-0/minicluster-data/master-0/data/instance:
uuid: "45bae8b55b8a4650b03a0a73db46f13f"
format_stamp: "Formatted at 2026-08-06 18:38:31 on dist-test-slave-1xrh"
server_key: "f8d449c12f115d704b4c10be382b6b2b"
server_key_iv: "3f425e217978757686702646e8380c4b"
server_key_version: "encryptionkey@0"
I20260806 18:38:31.524017 9937 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786041246094603-20232-0/minicluster-data/master-0/wal/instance:
uuid: "45bae8b55b8a4650b03a0a73db46f13f"
format_stamp: "Formatted at 2026-08-06 18:38:31 on dist-test-slave-1xrh"
server_key: "f8d449c12f115d704b4c10be382b6b2b"
server_key_iv: "3f425e217978757686702646e8380c4b"
server_key_version: "encryptionkey@0"
I20260806 18:38:31.527552 9937 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.000s
I20260806 18:38:31.529848 9952 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 18:38:31.530987 9937 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260806 18:38:31.531167 9937 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786041246094603-20232-0/minicluster-data/master-0/data,/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786041246094603-20232-0/minicluster-data/master-0/wal
uuid: "45bae8b55b8a4650b03a0a73db46f13f"
format_stamp: "Formatted at 2026-08-06 18:38:31 on dist-test-slave-1xrh"
server_key: "f8d449c12f115d704b4c10be382b6b2b"
server_key_iv: "3f425e217978757686702646e8380c4b"
server_key_version: "encryptionkey@0"
I20260806 18:38:31.531286 9937 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786041246094603-20232-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786041246094603-20232-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786041246094603-20232-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 18:38:31.535657 9937 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 18:38:31.538419 9937 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 18:38:31.538604 9937 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 18:38:31.545684 9937 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:34165
I20260806 18:38:31.545711 10006 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:34165 every 8 connection(s)
I20260806 18:38:31.546536 9937 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786041246094603-20232-0/minicluster-data/master-0/data/info.pb
I20260806 18:38:31.549144 10007 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 18:38:31.551642 20232 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJW69tn/build/debug/bin/kudu as pid 9937
I20260806 18:38:31.551791 20232 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786041246094603-20232-0/minicluster-data/master-0/wal/instance
I20260806 18:38:31.552136 20232 external_mini_cluster.cc:1428] Setting key d2fe63eb053b775a61663a9412014101
I20260806 18:38:31.555217 10007 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 45bae8b55b8a4650b03a0a73db46f13f: Bootstrap starting.
I20260806 18:38:31.557108 10012 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:38:31.553519 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.186:58917 (local address 127.0.0.1:34165)
0806 18:38:31.553981 (+ 462us) server_negotiation.cc:206] Beginning negotiation
0806 18:38:31.553996 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0806 18:38:31.554029 (+ 33us) server_negotiation.cc:407] Connection header received
0806 18:38:31.554622 (+ 593us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:38:31.554637 (+ 15us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:38:31.554910 (+ 273us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:38:31.555262 (+ 352us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:38:31.556008 (+ 746us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:38:31.556013 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:38:31.556024 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 18:38:31.556035 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:38:31.556087 (+ 52us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 18:38:31.556106 (+ 19us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:38:31.556254 (+ 148us) server_negotiation.cc:1035] Waiting for connection context
0806 18:38:31.556321 (+ 67us) server_negotiation.cc:299] Negotiation successful
0806 18:38:31.556478 (+ 157us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":317,"thread_start_us":135,"threads_started":1}
I20260806 18:38:31.557601 10007 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 45bae8b55b8a4650b03a0a73db46f13f: Neither blocks nor log segments found. Creating new log.
I20260806 18:38:31.558454 10007 log.cc:824] T 00000000000000000000000000000000 P 45bae8b55b8a4650b03a0a73db46f13f: Log is configured to *not* fsync() on all Append() calls
I20260806 18:38:31.559932 20232 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260806 18:38:31.560657 10007 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 45bae8b55b8a4650b03a0a73db46f13f: No bootstrap required, opened a new log
I20260806 18:38:31.563709 10012 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:38:31.562012 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.186:50993 (local address 127.0.0.1:34165)
0806 18:38:31.562139 (+ 127us) server_negotiation.cc:206] Beginning negotiation
0806 18:38:31.562143 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 18:38:31.562357 (+ 214us) server_negotiation.cc:407] Connection header received
0806 18:38:31.562516 (+ 159us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:38:31.562519 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:38:31.562555 (+ 36us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:38:31.562633 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:38:31.563339 (+ 706us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:38:31.563342 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:38:31.563344 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 18:38:31.563356 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:38:31.563380 (+ 24us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 18:38:31.563399 (+ 19us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:38:31.563488 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0806 18:38:31.563563 (+ 75us) server_negotiation.cc:299] Negotiation successful
0806 18:38:31.563601 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260806 18:38:31.564611 10007 raft_consensus.cc:348] T 00000000000000000000000000000000 P 45bae8b55b8a4650b03a0a73db46f13f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "45bae8b55b8a4650b03a0a73db46f13f" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34165 } }
I20260806 18:38:31.564828 10007 raft_consensus.cc:374] T 00000000000000000000000000000000 P 45bae8b55b8a4650b03a0a73db46f13f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 18:38:31.564900 10007 raft_consensus.cc:729] T 00000000000000000000000000000000 P 45bae8b55b8a4650b03a0a73db46f13f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 45bae8b55b8a4650b03a0a73db46f13f, State: Initialized, Role: FOLLOWER
I20260806 18:38:31.565331 10007 consensus_queue.cc:260] T 00000000000000000000000000000000 P 45bae8b55b8a4650b03a0a73db46f13f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "45bae8b55b8a4650b03a0a73db46f13f" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34165 } }
I20260806 18:38:31.565471 10007 raft_consensus.cc:388] T 00000000000000000000000000000000 P 45bae8b55b8a4650b03a0a73db46f13f [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 18:38:31.565559 10007 raft_consensus.cc:482] T 00000000000000000000000000000000 P 45bae8b55b8a4650b03a0a73db46f13f [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 18:38:31.565675 10007 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 45bae8b55b8a4650b03a0a73db46f13f [term 0 FOLLOWER]: Advancing to term 1
I20260806 18:38:31.566501 10007 raft_consensus.cc:504] T 00000000000000000000000000000000 P 45bae8b55b8a4650b03a0a73db46f13f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "45bae8b55b8a4650b03a0a73db46f13f" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34165 } }
I20260806 18:38:31.566823 10007 leader_election.cc:302] T 00000000000000000000000000000000 P 45bae8b55b8a4650b03a0a73db46f13f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 45bae8b55b8a4650b03a0a73db46f13f; no voters:
I20260806 18:38:31.567128 10007 leader_election.cc:288] T 00000000000000000000000000000000 P 45bae8b55b8a4650b03a0a73db46f13f [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 18:38:31.567222 10019 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 45bae8b55b8a4650b03a0a73db46f13f [term 1 FOLLOWER]: Leader election won for term 1
I20260806 18:38:31.567473 10019 raft_consensus.cc:686] T 00000000000000000000000000000000 P 45bae8b55b8a4650b03a0a73db46f13f [term 1 LEADER]: Becoming Leader. State: Replica: 45bae8b55b8a4650b03a0a73db46f13f, State: Running, Role: LEADER
I20260806 18:38:31.567831 10019 consensus_queue.cc:237] T 00000000000000000000000000000000 P 45bae8b55b8a4650b03a0a73db46f13f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "45bae8b55b8a4650b03a0a73db46f13f" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34165 } }
I20260806 18:38:31.568370 10007 sys_catalog.cc:564] T 00000000000000000000000000000000 P 45bae8b55b8a4650b03a0a73db46f13f [sys.catalog]: configured and running, proceeding with master startup.
I20260806 18:38:31.569474 10020 sys_catalog.cc:455] T 00000000000000000000000000000000 P 45bae8b55b8a4650b03a0a73db46f13f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "45bae8b55b8a4650b03a0a73db46f13f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "45bae8b55b8a4650b03a0a73db46f13f" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34165 } } }
I20260806 18:38:31.569612 10020 sys_catalog.cc:458] T 00000000000000000000000000000000 P 45bae8b55b8a4650b03a0a73db46f13f [sys.catalog]: This master's current role is: LEADER
I20260806 18:38:31.569947 10021 sys_catalog.cc:455] T 00000000000000000000000000000000 P 45bae8b55b8a4650b03a0a73db46f13f [sys.catalog]: SysCatalogTable state changed. Reason: New leader 45bae8b55b8a4650b03a0a73db46f13f. Latest consensus state: current_term: 1 leader_uuid: "45bae8b55b8a4650b03a0a73db46f13f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "45bae8b55b8a4650b03a0a73db46f13f" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34165 } } }
I20260806 18:38:31.570029 10021 sys_catalog.cc:458] T 00000000000000000000000000000000 P 45bae8b55b8a4650b03a0a73db46f13f [sys.catalog]: This master's current role is: LEADER
I20260806 18:38:31.570048 10029 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 18:38:31.572573 10029 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 18:38:31.577450 10029 catalog_manager.cc:1347] Generated new cluster ID: 814c2db951c44b87bb8aaaef591e1663
I20260806 18:38:31.577512 10029 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 18:38:31.610973 10029 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 18:38:31.611328 10029 catalog_manager.cc:1504] Loading token signing keys...
I20260806 18:38:31.622334 10029 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 45bae8b55b8a4650b03a0a73db46f13f: Generated new TSK 0
I20260806 18:38:31.623056 10029 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 18:38:31.629968 20232 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJW69tn/build/debug/bin/kudu with pid 9937
2026-08-06T18:38:31Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/8 (1418 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/9
Loading random data
Initializing database '/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786041246094603-20232-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 18:38:31 dist-test-slave-1xrh krb5kdc[10041](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 18:38:31 dist-test-slave-1xrh krb5kdc[10041](info): set up 2 sockets
Aug 06 18:38:31 dist-test-slave-1xrh krb5kdc[10041](info): commencing operation
krb5kdc: starting...
W20260806 18:38:32.688086 20232 mini_kdc.cc:121] Time spent starting KDC: real 1.036s user 0.004s sys 0.003s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786041246094603-20232-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786041246094603-20232-0/krb5kdc/test-user.keytab.
Aug 06 18:38:32 dist-test-slave-1xrh krb5kdc[10041](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041512, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T18:38:32Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T18:38:32Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786041246094603-20232-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786041246094603-20232-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260806 18:38:32.808130 20232 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJW69tn/build/debug/bin/kudu
/tmp/dist-test-taskJW69tn/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786041246094603-20232-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786041246094603-20232-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786041246094603-20232-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786041246094603-20232-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:39743
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786041246094603-20232-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:32911
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:39743
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786041246094603-20232-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786041246094603-20232-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786041246094603-20232-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 18:38:32.917110 10054 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 18:38:32.917343 10054 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 18:38:32.917398 10054 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 18:38:32.921211 10054 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 18:38:32.921277 10054 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 18:38:32.921296 10054 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 18:38:32.921311 10054 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 18:38:32.921326 10054 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 18:38:32.926443 10054 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:32911
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786041246094603-20232-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786041246094603-20232-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:39743
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786041246094603-20232-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:39743
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786041246094603-20232-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10054
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786041246094603-20232-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 17:59:13 UTC on ccycloud.quasar-wvsnzx.root.comops.site
build id 4571028
I20260806 18:38:32.927469 10054 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 18:38:32.928428 10054 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260806 18:38:32.936673 10054 server_base.cc:1034] running on GCE node
W20260806 18:38:32.936612 10060 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:38:32.936601 10061 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:38:32.936596 10063 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 18:38:32.937597 10054 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 18:38:32.938640 10054 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 18:38:32.939793 10054 hybrid_clock.cc:648] HybridClock initialized: now 1786041512939763 us; error 45 us; skew 500 ppm
Aug 06 18:38:32 dist-test-slave-1xrh krb5kdc[10041](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041512, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 18:38:32.942517 10054 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260806 18:38:32.943461 10054 webserver.cc:460] Webserver started at http://127.0.0.1:33271/ using document root <none> and password file <none>
I20260806 18:38:32.943950 10054 fs_manager.cc:359] Metadata directory not provided
I20260806 18:38:32.944010 10054 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 18:38:32.944161 10054 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 18:38:32.946029 10054 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786041246094603-20232-0/minicluster-data/master-0/data/instance:
uuid: "e7b5121cac854e4dba4b0c334df25e19"
format_stamp: "Formatted at 2026-08-06 18:38:32 on dist-test-slave-1xrh"
server_key: "69b7cfed660007627eb82af5a7888b03"
server_key_iv: "76b06710b623f0c9843f53d6c2386624"
server_key_version: "encryptionkey@0"
I20260806 18:38:32.946525 10054 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786041246094603-20232-0/minicluster-data/master-0/wal/instance:
uuid: "e7b5121cac854e4dba4b0c334df25e19"
format_stamp: "Formatted at 2026-08-06 18:38:32 on dist-test-slave-1xrh"
server_key: "69b7cfed660007627eb82af5a7888b03"
server_key_iv: "76b06710b623f0c9843f53d6c2386624"
server_key_version: "encryptionkey@0"
I20260806 18:38:32.950393 10054 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260806 18:38:32.952680 10070 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 18:38:32.953868 10054 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260806 18:38:32.953992 10054 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786041246094603-20232-0/minicluster-data/master-0/data,/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786041246094603-20232-0/minicluster-data/master-0/wal
uuid: "e7b5121cac854e4dba4b0c334df25e19"
format_stamp: "Formatted at 2026-08-06 18:38:32 on dist-test-slave-1xrh"
server_key: "69b7cfed660007627eb82af5a7888b03"
server_key_iv: "76b06710b623f0c9843f53d6c2386624"
server_key_version: "encryptionkey@0"
I20260806 18:38:32.954094 10054 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786041246094603-20232-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786041246094603-20232-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786041246094603-20232-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 18:38:32.972587 10054 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 18:38:32.975299 10054 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 18:38:32.975497 10054 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 18:38:32.982308 10054 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:39743
I20260806 18:38:32.982319 10122 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:39743 every 8 connection(s)
I20260806 18:38:32.983244 10054 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786041246094603-20232-0/minicluster-data/master-0/data/info.pb
I20260806 18:38:32.984005 20232 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJW69tn/build/debug/bin/kudu as pid 10054
I20260806 18:38:32.984143 20232 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786041246094603-20232-0/minicluster-data/master-0/wal/instance
I20260806 18:38:32.984469 20232 external_mini_cluster.cc:1428] Setting key 439de5c74c2a2d48549200df8da2a129
I20260806 18:38:32.986361 10123 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 18:38:32.992607 10123 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e7b5121cac854e4dba4b0c334df25e19: Bootstrap starting.
Aug 06 18:38:32 dist-test-slave-1xrh krb5kdc[10041](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041512, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260806 18:38:32.994731 10123 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P e7b5121cac854e4dba4b0c334df25e19: Neither blocks nor log segments found. Creating new log.
I20260806 18:38:32.995486 10123 log.cc:824] T 00000000000000000000000000000000 P e7b5121cac854e4dba4b0c334df25e19: Log is configured to *not* fsync() on all Append() calls
I20260806 18:38:32.997179 10123 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e7b5121cac854e4dba4b0c334df25e19: No bootstrap required, opened a new log
I20260806 18:38:32.998759 10126 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:38:32.985762 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.186:50745 (local address 127.0.0.1:39743)
0806 18:38:32.986164 (+ 402us) server_negotiation.cc:206] Beginning negotiation
0806 18:38:32.986188 (+ 24us) server_negotiation.cc:399] Waiting for connection header
0806 18:38:32.986221 (+ 33us) server_negotiation.cc:407] Connection header received
0806 18:38:32.986891 (+ 670us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:38:32.986920 (+ 29us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:38:32.987297 (+ 377us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:38:32.987557 (+ 260us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:38:32.988427 (+ 870us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:38:32.989401 (+ 974us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:38:32.990530 (+ 1129us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:38:32.991073 (+ 543us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:38:32.991267 (+ 194us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:38:32.993995 (+ 2728us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:38:32.994000 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:38:32.994012 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 18:38:32.994037 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:38:32.995798 (+ 1761us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:38:32.996300 (+ 502us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:38:32.996305 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:38:32.996309 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:38:32.996413 (+ 104us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:38:32.996752 (+ 339us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:38:32.996755 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:38:32.996757 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:38:32.997119 (+ 362us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:38:32.997326 (+ 207us) server_negotiation.cc:1035] Waiting for connection context
0806 18:38:32.997690 (+ 364us) server_negotiation.cc:299] Negotiation successful
0806 18:38:32.997892 (+ 202us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":270,"thread_start_us":152,"threads_started":1}
I20260806 18:38:33.000393 10123 raft_consensus.cc:348] T 00000000000000000000000000000000 P e7b5121cac854e4dba4b0c334df25e19 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e7b5121cac854e4dba4b0c334df25e19" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39743 } }
I20260806 18:38:33.000730 10123 raft_consensus.cc:374] T 00000000000000000000000000000000 P e7b5121cac854e4dba4b0c334df25e19 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 18:38:33.000815 10123 raft_consensus.cc:729] T 00000000000000000000000000000000 P e7b5121cac854e4dba4b0c334df25e19 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e7b5121cac854e4dba4b0c334df25e19, State: Initialized, Role: FOLLOWER
I20260806 18:38:33.001240 10123 consensus_queue.cc:260] T 00000000000000000000000000000000 P e7b5121cac854e4dba4b0c334df25e19 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e7b5121cac854e4dba4b0c334df25e19" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39743 } }
I20260806 18:38:33.001398 10123 raft_consensus.cc:388] T 00000000000000000000000000000000 P e7b5121cac854e4dba4b0c334df25e19 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 18:38:33.001472 10123 raft_consensus.cc:482] T 00000000000000000000000000000000 P e7b5121cac854e4dba4b0c334df25e19 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 18:38:33.001571 10123 raft_consensus.cc:3037] T 00000000000000000000000000000000 P e7b5121cac854e4dba4b0c334df25e19 [term 0 FOLLOWER]: Advancing to term 1
I20260806 18:38:33.001718 20232 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260806 18:38:33.002504 10123 raft_consensus.cc:504] T 00000000000000000000000000000000 P e7b5121cac854e4dba4b0c334df25e19 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e7b5121cac854e4dba4b0c334df25e19" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39743 } }
I20260806 18:38:33.002851 10123 leader_election.cc:302] T 00000000000000000000000000000000 P e7b5121cac854e4dba4b0c334df25e19 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: e7b5121cac854e4dba4b0c334df25e19; no voters:
I20260806 18:38:33.003233 10123 leader_election.cc:288] T 00000000000000000000000000000000 P e7b5121cac854e4dba4b0c334df25e19 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 18:38:33.003301 10132 raft_consensus.cc:2781] T 00000000000000000000000000000000 P e7b5121cac854e4dba4b0c334df25e19 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 18:38:33.003500 10132 raft_consensus.cc:686] T 00000000000000000000000000000000 P e7b5121cac854e4dba4b0c334df25e19 [term 1 LEADER]: Becoming Leader. State: Replica: e7b5121cac854e4dba4b0c334df25e19, State: Running, Role: LEADER
I20260806 18:38:33.003850 10132 consensus_queue.cc:237] T 00000000000000000000000000000000 P e7b5121cac854e4dba4b0c334df25e19 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e7b5121cac854e4dba4b0c334df25e19" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39743 } }
I20260806 18:38:33.004469 10123 sys_catalog.cc:564] T 00000000000000000000000000000000 P e7b5121cac854e4dba4b0c334df25e19 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 18:38:33.005499 10134 sys_catalog.cc:455] T 00000000000000000000000000000000 P e7b5121cac854e4dba4b0c334df25e19 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "e7b5121cac854e4dba4b0c334df25e19" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e7b5121cac854e4dba4b0c334df25e19" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39743 } } }
I20260806 18:38:33.005625 10134 sys_catalog.cc:458] T 00000000000000000000000000000000 P e7b5121cac854e4dba4b0c334df25e19 [sys.catalog]: This master's current role is: LEADER
I20260806 18:38:33.005976 10133 sys_catalog.cc:455] T 00000000000000000000000000000000 P e7b5121cac854e4dba4b0c334df25e19 [sys.catalog]: SysCatalogTable state changed. Reason: New leader e7b5121cac854e4dba4b0c334df25e19. Latest consensus state: current_term: 1 leader_uuid: "e7b5121cac854e4dba4b0c334df25e19" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e7b5121cac854e4dba4b0c334df25e19" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39743 } } }
I20260806 18:38:33.006043 10142 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 18:38:33.006057 10133 sys_catalog.cc:458] T 00000000000000000000000000000000 P e7b5121cac854e4dba4b0c334df25e19 [sys.catalog]: This master's current role is: LEADER
I20260806 18:38:33.008699 10142 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 18:38:33.014333 10142 catalog_manager.cc:1347] Generated new cluster ID: 25ffba1e08a34b8ea899735d1cb19dbd
I20260806 18:38:33.014390 10142 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 18:38:33.014617 10126 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:38:33.003805 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.186:59135 (local address 127.0.0.1:39743)
0806 18:38:33.003906 (+ 101us) server_negotiation.cc:206] Beginning negotiation
0806 18:38:33.003910 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 18:38:33.004434 (+ 524us) server_negotiation.cc:407] Connection header received
0806 18:38:33.004576 (+ 142us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:38:33.004580 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:38:33.004621 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:38:33.004704 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:38:33.007383 (+ 2679us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:38:33.007874 (+ 491us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:38:33.009026 (+ 1152us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:38:33.009405 (+ 379us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:38:33.009466 (+ 61us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:38:33.011054 (+ 1588us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:38:33.011058 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:38:33.011060 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 18:38:33.011084 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:38:33.012830 (+ 1746us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:38:33.013305 (+ 475us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:38:33.013309 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:38:33.013311 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:38:33.013395 (+ 84us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:38:33.013737 (+ 342us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:38:33.013741 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:38:33.013742 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:38:33.013915 (+ 173us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:38:33.014065 (+ 150us) server_negotiation.cc:1035] Waiting for connection context
0806 18:38:33.014344 (+ 279us) server_negotiation.cc:299] Negotiation successful
0806 18:38:33.014446 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260806 18:38:33.047411 10142 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 18:38:33.048629 10142 catalog_manager.cc:1504] Loading token signing keys...
I20260806 18:38:33.054776 10142 catalog_manager.cc:5868] T 00000000000000000000000000000000 P e7b5121cac854e4dba4b0c334df25e19: Generated new TSK 0
I20260806 18:38:33.055461 10142 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 18:38:33.060685 20232 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJW69tn/build/debug/bin/kudu with pid 10054
W20260806 18:38:33.068790 10039 connection.cc:535] client connection to 127.0.0.1:39743 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:39743): Connection reset by peer (error 104)
2026-08-06T18:38:33Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/9 (1430 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/10
Loading random data
Initializing database '/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786041246094603-20232-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 18:38:33 dist-test-slave-1xrh krb5kdc[10156](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 18:38:33 dist-test-slave-1xrh krb5kdc[10156](info): set up 2 sockets
Aug 06 18:38:33 dist-test-slave-1xrh krb5kdc[10156](info): commencing operation
krb5kdc: starting...
W20260806 18:38:34.108309 20232 mini_kdc.cc:121] Time spent starting KDC: real 1.025s user 0.005s sys 0.001s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786041246094603-20232-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786041246094603-20232-0/krb5kdc/test-user.keytab.
Aug 06 18:38:34 dist-test-slave-1xrh krb5kdc[10156](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041514, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T18:38:34Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T18:38:34Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786041246094603-20232-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786041246094603-20232-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260806 18:38:34.225646 20232 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJW69tn/build/debug/bin/kudu
/tmp/dist-test-taskJW69tn/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786041246094603-20232-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786041246094603-20232-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786041246094603-20232-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786041246094603-20232-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:39523
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786041246094603-20232-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:36645
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:39523
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786041246094603-20232-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786041246094603-20232-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786041246094603-20232-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 18:38:34.332412 10170 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 18:38:34.332679 10170 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 18:38:34.332762 10170 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 18:38:34.336724 10170 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 18:38:34.336805 10170 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 18:38:34.336845 10170 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 18:38:34.336910 10170 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 18:38:34.336993 10170 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 18:38:34.342106 10170 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:36645
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786041246094603-20232-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786041246094603-20232-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:39523
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786041246094603-20232-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:39523
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786041246094603-20232-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10170
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786041246094603-20232-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 17:59:13 UTC on ccycloud.quasar-wvsnzx.root.comops.site
build id 4571028
I20260806 18:38:34.343082 10170 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 18:38:34.344085 10170 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 18:38:34.352448 10176 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:38:34.352459 10175 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:38:34.352468 10178 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 18:38:34.352778 10170 server_base.cc:1034] running on GCE node
I20260806 18:38:34.353302 10170 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 18:38:34.354331 10170 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 18:38:34.355522 10170 hybrid_clock.cc:648] HybridClock initialized: now 1786041514355497 us; error 42 us; skew 500 ppm
Aug 06 18:38:34 dist-test-slave-1xrh krb5kdc[10156](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041514, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 18:38:34.358160 10170 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260806 18:38:34.358928 10170 webserver.cc:460] Webserver started at http://127.0.0.1:45133/ using document root <none> and password file <none>
I20260806 18:38:34.359484 10170 fs_manager.cc:359] Metadata directory not provided
I20260806 18:38:34.359558 10170 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 18:38:34.359710 10170 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 18:38:34.361675 10170 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786041246094603-20232-0/minicluster-data/master-0/data/instance:
uuid: "1490806e1a674a278e5ae227bc9ced92"
format_stamp: "Formatted at 2026-08-06 18:38:34 on dist-test-slave-1xrh"
server_key: "abf4eab476509807cb01223402c2a61f"
server_key_iv: "9a41706eeb0f0ce16ba41b1684d868e5"
server_key_version: "encryptionkey@0"
I20260806 18:38:34.362169 10170 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786041246094603-20232-0/minicluster-data/master-0/wal/instance:
uuid: "1490806e1a674a278e5ae227bc9ced92"
format_stamp: "Formatted at 2026-08-06 18:38:34 on dist-test-slave-1xrh"
server_key: "abf4eab476509807cb01223402c2a61f"
server_key_iv: "9a41706eeb0f0ce16ba41b1684d868e5"
server_key_version: "encryptionkey@0"
I20260806 18:38:34.365751 10170 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.006s sys 0.000s
I20260806 18:38:34.368036 10185 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 18:38:34.369184 10170 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260806 18:38:34.369307 10170 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786041246094603-20232-0/minicluster-data/master-0/data,/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786041246094603-20232-0/minicluster-data/master-0/wal
uuid: "1490806e1a674a278e5ae227bc9ced92"
format_stamp: "Formatted at 2026-08-06 18:38:34 on dist-test-slave-1xrh"
server_key: "abf4eab476509807cb01223402c2a61f"
server_key_iv: "9a41706eeb0f0ce16ba41b1684d868e5"
server_key_version: "encryptionkey@0"
I20260806 18:38:34.369410 10170 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786041246094603-20232-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786041246094603-20232-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786041246094603-20232-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 18:38:34.392546 10170 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 18:38:34.395323 10170 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 18:38:34.395536 10170 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 18:38:34.402472 10170 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:39523
I20260806 18:38:34.402499 10238 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:39523 every 8 connection(s)
I20260806 18:38:34.403414 10170 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786041246094603-20232-0/minicluster-data/master-0/data/info.pb
I20260806 18:38:34.406028 10239 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 18:38:34.411510 20232 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJW69tn/build/debug/bin/kudu as pid 10170
I20260806 18:38:34.411511 10239 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1490806e1a674a278e5ae227bc9ced92: Bootstrap starting.
I20260806 18:38:34.411618 20232 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786041246094603-20232-0/minicluster-data/master-0/wal/instance
I20260806 18:38:34.411895 20232 external_mini_cluster.cc:1428] Setting key 81dec09e5c7ab22de12b081e28e88c35
I20260806 18:38:34.414084 10239 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 1490806e1a674a278e5ae227bc9ced92: Neither blocks nor log segments found. Creating new log.
I20260806 18:38:34.414794 10239 log.cc:824] T 00000000000000000000000000000000 P 1490806e1a674a278e5ae227bc9ced92: Log is configured to *not* fsync() on all Append() calls
I20260806 18:38:34.416618 10239 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1490806e1a674a278e5ae227bc9ced92: No bootstrap required, opened a new log
I20260806 18:38:34.420397 10243 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:38:34.413187 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.186:43225 (local address 127.0.0.1:39523)
0806 18:38:34.413611 (+ 424us) server_negotiation.cc:206] Beginning negotiation
0806 18:38:34.413625 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0806 18:38:34.413658 (+ 33us) server_negotiation.cc:407] Connection header received
0806 18:38:34.414315 (+ 657us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:38:34.414332 (+ 17us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:38:34.414655 (+ 323us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:38:34.414998 (+ 343us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:38:34.415413 (+ 415us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:38:34.416353 (+ 940us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:38:34.417314 (+ 961us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:38:34.417749 (+ 435us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:38:34.417914 (+ 165us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:38:34.418790 (+ 876us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:38:34.418795 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:38:34.418807 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 18:38:34.419217 (+ 410us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:38:34.419271 (+ 54us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 18:38:34.419294 (+ 23us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:38:34.419437 (+ 143us) server_negotiation.cc:1035] Waiting for connection context
0806 18:38:34.419609 (+ 172us) server_negotiation.cc:299] Negotiation successful
0806 18:38:34.419738 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":286,"thread_start_us":136,"threads_started":1}
I20260806 18:38:34.420301 10239 raft_consensus.cc:348] T 00000000000000000000000000000000 P 1490806e1a674a278e5ae227bc9ced92 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1490806e1a674a278e5ae227bc9ced92" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39523 } }
I20260806 18:38:34.420524 10239 raft_consensus.cc:374] T 00000000000000000000000000000000 P 1490806e1a674a278e5ae227bc9ced92 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 18:38:34.420583 10239 raft_consensus.cc:729] T 00000000000000000000000000000000 P 1490806e1a674a278e5ae227bc9ced92 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1490806e1a674a278e5ae227bc9ced92, State: Initialized, Role: FOLLOWER
I20260806 18:38:34.421067 10239 consensus_queue.cc:260] T 00000000000000000000000000000000 P 1490806e1a674a278e5ae227bc9ced92 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1490806e1a674a278e5ae227bc9ced92" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39523 } }
I20260806 18:38:34.421221 10239 raft_consensus.cc:388] T 00000000000000000000000000000000 P 1490806e1a674a278e5ae227bc9ced92 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 18:38:34.421285 10239 raft_consensus.cc:482] T 00000000000000000000000000000000 P 1490806e1a674a278e5ae227bc9ced92 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 18:38:34.421379 10239 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 1490806e1a674a278e5ae227bc9ced92 [term 0 FOLLOWER]: Advancing to term 1
I20260806 18:38:34.422309 10239 raft_consensus.cc:504] T 00000000000000000000000000000000 P 1490806e1a674a278e5ae227bc9ced92 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1490806e1a674a278e5ae227bc9ced92" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39523 } }
I20260806 18:38:34.422679 10239 leader_election.cc:302] T 00000000000000000000000000000000 P 1490806e1a674a278e5ae227bc9ced92 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 1490806e1a674a278e5ae227bc9ced92; no voters:
I20260806 18:38:34.423020 10239 leader_election.cc:288] T 00000000000000000000000000000000 P 1490806e1a674a278e5ae227bc9ced92 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 18:38:34.423110 10245 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 1490806e1a674a278e5ae227bc9ced92 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 18:38:34.423348 20232 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260806 18:38:34.423398 10245 raft_consensus.cc:686] T 00000000000000000000000000000000 P 1490806e1a674a278e5ae227bc9ced92 [term 1 LEADER]: Becoming Leader. State: Replica: 1490806e1a674a278e5ae227bc9ced92, State: Running, Role: LEADER
I20260806 18:38:34.423671 10245 consensus_queue.cc:237] T 00000000000000000000000000000000 P 1490806e1a674a278e5ae227bc9ced92 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1490806e1a674a278e5ae227bc9ced92" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39523 } }
I20260806 18:38:34.424338 10239 sys_catalog.cc:564] T 00000000000000000000000000000000 P 1490806e1a674a278e5ae227bc9ced92 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 18:38:34.424590 10245 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1490806e1a674a278e5ae227bc9ced92 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 1490806e1a674a278e5ae227bc9ced92. Latest consensus state: current_term: 1 leader_uuid: "1490806e1a674a278e5ae227bc9ced92" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1490806e1a674a278e5ae227bc9ced92" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39523 } } }
I20260806 18:38:34.424703 10245 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1490806e1a674a278e5ae227bc9ced92 [sys.catalog]: This master's current role is: LEADER
I20260806 18:38:34.425010 10246 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1490806e1a674a278e5ae227bc9ced92 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "1490806e1a674a278e5ae227bc9ced92" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1490806e1a674a278e5ae227bc9ced92" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39523 } } }
I20260806 18:38:34.425099 10246 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1490806e1a674a278e5ae227bc9ced92 [sys.catalog]: This master's current role is: LEADER
I20260806 18:38:34.425122 10253 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 18:38:34.428253 10253 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 18:38:34.430645 10243 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:38:34.425853 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.186:60991 (local address 127.0.0.1:39523)
0806 18:38:34.425958 (+ 105us) server_negotiation.cc:206] Beginning negotiation
0806 18:38:34.425962 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 18:38:34.426195 (+ 233us) server_negotiation.cc:407] Connection header received
0806 18:38:34.426374 (+ 179us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:38:34.426380 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:38:34.426436 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:38:34.426523 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:38:34.426838 (+ 315us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:38:34.427589 (+ 751us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:38:34.428870 (+ 1281us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:38:34.429157 (+ 287us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:38:34.429268 (+ 111us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:38:34.430134 (+ 866us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:38:34.430138 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:38:34.430140 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 18:38:34.430152 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:38:34.430178 (+ 26us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 18:38:34.430190 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:38:34.430274 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0806 18:38:34.430472 (+ 198us) server_negotiation.cc:299] Negotiation successful
0806 18:38:34.430505 (+ 33us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260806 18:38:34.433876 10253 catalog_manager.cc:1347] Generated new cluster ID: 8201b8d6f4c646589fe0a2e4a2e6ca30
I20260806 18:38:34.433946 10253 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 18:38:34.438808 10253 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 18:38:34.439661 10253 catalog_manager.cc:1504] Loading token signing keys...
I20260806 18:38:34.447516 10253 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 1490806e1a674a278e5ae227bc9ced92: Generated new TSK 0
I20260806 18:38:34.448148 10253 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 18:38:34.452661 20232 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJW69tn/build/debug/bin/kudu with pid 10170
W20260806 18:38:34.461174 10153 connection.cc:535] client connection to 127.0.0.1:39523 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:39523): Connection reset by peer (error 104)
2026-08-06T18:38:34Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/10 (1392 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/11
Loading random data
Initializing database '/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786041246094603-20232-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 18:38:34 dist-test-slave-1xrh krb5kdc[10271](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 18:38:34 dist-test-slave-1xrh krb5kdc[10271](info): set up 2 sockets
Aug 06 18:38:34 dist-test-slave-1xrh krb5kdc[10271](info): commencing operation
krb5kdc: starting...
W20260806 18:38:35.516076 20232 mini_kdc.cc:121] Time spent starting KDC: real 1.040s user 0.003s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786041246094603-20232-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786041246094603-20232-0/krb5kdc/test-user.keytab.
Aug 06 18:38:35 dist-test-slave-1xrh krb5kdc[10271](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041515, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T18:38:35Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T18:38:35Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786041246094603-20232-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786041246094603-20232-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260806 18:38:35.636099 20232 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJW69tn/build/debug/bin/kudu
/tmp/dist-test-taskJW69tn/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786041246094603-20232-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786041246094603-20232-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786041246094603-20232-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786041246094603-20232-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:45559
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786041246094603-20232-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:33973
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:45559
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786041246094603-20232-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786041246094603-20232-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786041246094603-20232-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 18:38:35.742825 10284 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 18:38:35.743078 10284 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 18:38:35.743168 10284 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 18:38:35.747067 10284 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 18:38:35.747115 10284 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 18:38:35.747169 10284 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 18:38:35.747186 10284 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 18:38:35.747201 10284 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 18:38:35.752293 10284 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:33973
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786041246094603-20232-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786041246094603-20232-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:45559
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786041246094603-20232-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:45559
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786041246094603-20232-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10284
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786041246094603-20232-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 17:59:13 UTC on ccycloud.quasar-wvsnzx.root.comops.site
build id 4571028
I20260806 18:38:35.753252 10284 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 18:38:35.754177 10284 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 18:38:35.762323 10289 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:38:35.762419 10290 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:38:35.762351 10292 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 18:38:35.762600 10284 server_base.cc:1034] running on GCE node
I20260806 18:38:35.763288 10284 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 18:38:35.764369 10284 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 18:38:35.765538 10284 hybrid_clock.cc:648] HybridClock initialized: now 1786041515765512 us; error 39 us; skew 500 ppm
Aug 06 18:38:35 dist-test-slave-1xrh krb5kdc[10271](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041515, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 18:38:35.768249 10284 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260806 18:38:35.769096 10284 webserver.cc:460] Webserver started at http://127.0.0.1:34727/ using document root <none> and password file <none>
I20260806 18:38:35.769625 10284 fs_manager.cc:359] Metadata directory not provided
I20260806 18:38:35.769708 10284 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 18:38:35.769929 10284 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 18:38:35.771968 10284 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786041246094603-20232-0/minicluster-data/master-0/data/instance:
uuid: "61491c296abb433e94eb23658d69f64b"
format_stamp: "Formatted at 2026-08-06 18:38:35 on dist-test-slave-1xrh"
server_key: "d7da33efa5be0058e6fffcfc1ddb5525"
server_key_iv: "2b6ba45c6c33193e56ac115462779db6"
server_key_version: "encryptionkey@0"
I20260806 18:38:35.772512 10284 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786041246094603-20232-0/minicluster-data/master-0/wal/instance:
uuid: "61491c296abb433e94eb23658d69f64b"
format_stamp: "Formatted at 2026-08-06 18:38:35 on dist-test-slave-1xrh"
server_key: "d7da33efa5be0058e6fffcfc1ddb5525"
server_key_iv: "2b6ba45c6c33193e56ac115462779db6"
server_key_version: "encryptionkey@0"
I20260806 18:38:35.776149 10284 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20260806 18:38:35.778294 10300 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 18:38:35.779433 10284 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260806 18:38:35.779577 10284 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786041246094603-20232-0/minicluster-data/master-0/data,/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786041246094603-20232-0/minicluster-data/master-0/wal
uuid: "61491c296abb433e94eb23658d69f64b"
format_stamp: "Formatted at 2026-08-06 18:38:35 on dist-test-slave-1xrh"
server_key: "d7da33efa5be0058e6fffcfc1ddb5525"
server_key_iv: "2b6ba45c6c33193e56ac115462779db6"
server_key_version: "encryptionkey@0"
I20260806 18:38:35.779680 10284 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786041246094603-20232-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786041246094603-20232-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786041246094603-20232-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 18:38:35.784191 10284 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 18:38:35.786867 10284 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 18:38:35.787070 10284 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 18:38:35.794216 10284 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:45559
I20260806 18:38:35.794245 10356 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:45559 every 8 connection(s)
I20260806 18:38:35.795070 10284 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786041246094603-20232-0/minicluster-data/master-0/data/info.pb
I20260806 18:38:35.797608 10358 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 18:38:35.801903 20232 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJW69tn/build/debug/bin/kudu as pid 10284
I20260806 18:38:35.802055 20232 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786041246094603-20232-0/minicluster-data/master-0/wal/instance
I20260806 18:38:35.802341 20232 external_mini_cluster.cc:1428] Setting key fdf019c58f942a72ccd5d6d637f17f0f
I20260806 18:38:35.803056 10358 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 61491c296abb433e94eb23658d69f64b: Bootstrap starting.
I20260806 18:38:35.805672 10358 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 61491c296abb433e94eb23658d69f64b: Neither blocks nor log segments found. Creating new log.
I20260806 18:38:35.806573 10358 log.cc:824] T 00000000000000000000000000000000 P 61491c296abb433e94eb23658d69f64b: Log is configured to *not* fsync() on all Append() calls
I20260806 18:38:35.807864 10361 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:38:35.803743 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.186:33635 (local address 127.0.0.1:45559)
0806 18:38:35.804202 (+ 459us) server_negotiation.cc:206] Beginning negotiation
0806 18:38:35.804223 (+ 21us) server_negotiation.cc:399] Waiting for connection header
0806 18:38:35.804266 (+ 43us) server_negotiation.cc:407] Connection header received
0806 18:38:35.805057 (+ 791us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:38:35.805079 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:38:35.805413 (+ 334us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:38:35.805777 (+ 364us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:38:35.806555 (+ 778us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:38:35.806561 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:38:35.806575 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 18:38:35.806590 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:38:35.806659 (+ 69us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 18:38:35.806688 (+ 29us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:38:35.806793 (+ 105us) server_negotiation.cc:1035] Waiting for connection context
0806 18:38:35.806951 (+ 158us) server_negotiation.cc:299] Negotiation successful
0806 18:38:35.807114 (+ 163us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":283,"thread_start_us":126,"threads_started":1}
I20260806 18:38:35.808722 10358 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 61491c296abb433e94eb23658d69f64b: No bootstrap required, opened a new log
I20260806 18:38:35.810591 20232 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260806 18:38:35.812719 10358 raft_consensus.cc:348] T 00000000000000000000000000000000 P 61491c296abb433e94eb23658d69f64b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "61491c296abb433e94eb23658d69f64b" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45559 } }
I20260806 18:38:35.813014 10358 raft_consensus.cc:374] T 00000000000000000000000000000000 P 61491c296abb433e94eb23658d69f64b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 18:38:35.813098 10358 raft_consensus.cc:729] T 00000000000000000000000000000000 P 61491c296abb433e94eb23658d69f64b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 61491c296abb433e94eb23658d69f64b, State: Initialized, Role: FOLLOWER
I20260806 18:38:35.813584 10358 consensus_queue.cc:260] T 00000000000000000000000000000000 P 61491c296abb433e94eb23658d69f64b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "61491c296abb433e94eb23658d69f64b" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45559 } }
I20260806 18:38:35.813761 10358 raft_consensus.cc:388] T 00000000000000000000000000000000 P 61491c296abb433e94eb23658d69f64b [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 18:38:35.813848 10358 raft_consensus.cc:482] T 00000000000000000000000000000000 P 61491c296abb433e94eb23658d69f64b [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 18:38:35.813982 10358 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 61491c296abb433e94eb23658d69f64b [term 0 FOLLOWER]: Advancing to term 1
I20260806 18:38:35.814380 10361 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:38:35.812680 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.186:43755 (local address 127.0.0.1:45559)
0806 18:38:35.812786 (+ 106us) server_negotiation.cc:206] Beginning negotiation
0806 18:38:35.812792 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 18:38:35.812990 (+ 198us) server_negotiation.cc:407] Connection header received
0806 18:38:35.813132 (+ 142us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:38:35.813135 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:38:35.813165 (+ 30us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:38:35.813240 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:38:35.813962 (+ 722us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:38:35.813965 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:38:35.813967 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 18:38:35.813978 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:38:35.814002 (+ 24us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 18:38:35.814013 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:38:35.814108 (+ 95us) server_negotiation.cc:1035] Waiting for connection context
0806 18:38:35.814243 (+ 135us) server_negotiation.cc:299] Negotiation successful
0806 18:38:35.814280 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260806 18:38:35.815075 10358 raft_consensus.cc:504] T 00000000000000000000000000000000 P 61491c296abb433e94eb23658d69f64b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "61491c296abb433e94eb23658d69f64b" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45559 } }
I20260806 18:38:35.815411 10358 leader_election.cc:302] T 00000000000000000000000000000000 P 61491c296abb433e94eb23658d69f64b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 61491c296abb433e94eb23658d69f64b; no voters:
I20260806 18:38:35.815691 10358 leader_election.cc:288] T 00000000000000000000000000000000 P 61491c296abb433e94eb23658d69f64b [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 18:38:35.815798 10369 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 61491c296abb433e94eb23658d69f64b [term 1 FOLLOWER]: Leader election won for term 1
I20260806 18:38:35.816039 10369 raft_consensus.cc:686] T 00000000000000000000000000000000 P 61491c296abb433e94eb23658d69f64b [term 1 LEADER]: Becoming Leader. State: Replica: 61491c296abb433e94eb23658d69f64b, State: Running, Role: LEADER
I20260806 18:38:35.816378 10369 consensus_queue.cc:237] T 00000000000000000000000000000000 P 61491c296abb433e94eb23658d69f64b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "61491c296abb433e94eb23658d69f64b" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45559 } }
I20260806 18:38:35.816879 10358 sys_catalog.cc:564] T 00000000000000000000000000000000 P 61491c296abb433e94eb23658d69f64b [sys.catalog]: configured and running, proceeding with master startup.
I20260806 18:38:35.817960 10371 sys_catalog.cc:455] T 00000000000000000000000000000000 P 61491c296abb433e94eb23658d69f64b [sys.catalog]: SysCatalogTable state changed. Reason: New leader 61491c296abb433e94eb23658d69f64b. Latest consensus state: current_term: 1 leader_uuid: "61491c296abb433e94eb23658d69f64b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "61491c296abb433e94eb23658d69f64b" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45559 } } }
I20260806 18:38:35.818079 10371 sys_catalog.cc:458] T 00000000000000000000000000000000 P 61491c296abb433e94eb23658d69f64b [sys.catalog]: This master's current role is: LEADER
I20260806 18:38:35.818442 10370 sys_catalog.cc:455] T 00000000000000000000000000000000 P 61491c296abb433e94eb23658d69f64b [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "61491c296abb433e94eb23658d69f64b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "61491c296abb433e94eb23658d69f64b" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45559 } } }
I20260806 18:38:35.818538 10370 sys_catalog.cc:458] T 00000000000000000000000000000000 P 61491c296abb433e94eb23658d69f64b [sys.catalog]: This master's current role is: LEADER
I20260806 18:38:35.818682 10378 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 18:38:35.821679 10378 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 18:38:35.826720 10378 catalog_manager.cc:1347] Generated new cluster ID: 0ef23c4935134feeb2c197238413d1e0
I20260806 18:38:35.826781 10378 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 18:38:35.847411 10378 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 18:38:35.847735 10378 catalog_manager.cc:1504] Loading token signing keys...
I20260806 18:38:35.854655 10378 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 61491c296abb433e94eb23658d69f64b: Generated new TSK 0
I20260806 18:38:35.855360 10378 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 18:38:35.858664 20232 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJW69tn/build/debug/bin/kudu with pid 10284
2026-08-06T18:38:35Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/11 (1405 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/12
Loading random data
Initializing database '/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786041246094603-20232-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 18:38:35 dist-test-slave-1xrh krb5kdc[10390](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 18:38:35 dist-test-slave-1xrh krb5kdc[10390](info): set up 2 sockets
Aug 06 18:38:35 dist-test-slave-1xrh krb5kdc[10390](info): commencing operation
krb5kdc: starting...
W20260806 18:38:36.960539 20232 mini_kdc.cc:121] Time spent starting KDC: real 1.080s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786041246094603-20232-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786041246094603-20232-0/krb5kdc/test-user.keytab.
Aug 06 18:38:37 dist-test-slave-1xrh krb5kdc[10390](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041517, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T18:38:37Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T18:38:37Z Disabled control of system clock
WARNING: no policy specified for kudu/127.19.194.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.19.194.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.19.194.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786041246094603-20232-0/krb5kdc/kudu_127.19.194.62.keytab.
Entry for principal kudu/127.19.194.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786041246094603-20232-0/krb5kdc/kudu_127.19.194.62.keytab.
I20260806 18:38:37.084045 20232 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJW69tn/build/debug/bin/kudu
/tmp/dist-test-taskJW69tn/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786041246094603-20232-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786041246094603-20232-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786041246094603-20232-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786041246094603-20232-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.19.194.62:42545
--webserver_interface=127.19.194.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786041246094603-20232-0/krb5kdc/kudu_127.19.194.62.keytab
--principal=kudu/127.19.194.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.19.194.20:36297
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.19.194.62:42545
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786041246094603-20232-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786041246094603-20232-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786041246094603-20232-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 18:38:37.195827 10404 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 18:38:37.196069 10404 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 18:38:37.196110 10404 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 18:38:37.200047 10404 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 18:38:37.200093 10404 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 18:38:37.200110 10404 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 18:38:37.200142 10404 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 18:38:37.200157 10404 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 18:38:37.205188 10404 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.194.20:36297
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786041246094603-20232-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786041246094603-20232-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.19.194.62:42545
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786041246094603-20232-0/krb5kdc/kudu_127.19.194.62.keytab
--principal=kudu/127.19.194.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.19.194.62:42545
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786041246094603-20232-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.19.194.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10404
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786041246094603-20232-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 17:59:13 UTC on ccycloud.quasar-wvsnzx.root.comops.site
build id 4571028
I20260806 18:38:37.206075 10404 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 18:38:37.206938 10404 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 18:38:37.215088 10409 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:38:37.215117 10412 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:38:37.215140 10410 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 18:38:37.215242 10404 server_base.cc:1034] running on GCE node
I20260806 18:38:37.215883 10404 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 18:38:37.216866 10404 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 18:38:37.218061 10404 hybrid_clock.cc:648] HybridClock initialized: now 1786041517218043 us; error 41 us; skew 500 ppm
Aug 06 18:38:37 dist-test-slave-1xrh krb5kdc[10390](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041517, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.194.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 18:38:37.220748 10404 init.cc:377] Logged in from keytab as kudu/127.19.194.62@KRBTEST.COM (short username kudu)
I20260806 18:38:37.221506 10404 webserver.cc:460] Webserver started at http://127.19.194.62:33005/ using document root <none> and password file <none>
I20260806 18:38:37.221987 10404 fs_manager.cc:359] Metadata directory not provided
I20260806 18:38:37.222041 10404 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 18:38:37.222195 10404 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 18:38:37.224153 10404 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786041246094603-20232-0/minicluster-data/master-0/data/instance:
uuid: "ec530270436c4ef4bb785ecc40919d71"
format_stamp: "Formatted at 2026-08-06 18:38:37 on dist-test-slave-1xrh"
server_key: "ed4152e46c737d47180a482f6d36f05d"
server_key_iv: "5c8a983f88a3f8ffb0aa1033f1cc24e1"
server_key_version: "encryptionkey@0"
I20260806 18:38:37.224645 10404 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786041246094603-20232-0/minicluster-data/master-0/wal/instance:
uuid: "ec530270436c4ef4bb785ecc40919d71"
format_stamp: "Formatted at 2026-08-06 18:38:37 on dist-test-slave-1xrh"
server_key: "ed4152e46c737d47180a482f6d36f05d"
server_key_iv: "5c8a983f88a3f8ffb0aa1033f1cc24e1"
server_key_version: "encryptionkey@0"
I20260806 18:38:37.228114 10404 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20260806 18:38:37.230403 10419 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 18:38:37.231796 10404 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260806 18:38:37.231983 10404 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786041246094603-20232-0/minicluster-data/master-0/data,/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786041246094603-20232-0/minicluster-data/master-0/wal
uuid: "ec530270436c4ef4bb785ecc40919d71"
format_stamp: "Formatted at 2026-08-06 18:38:37 on dist-test-slave-1xrh"
server_key: "ed4152e46c737d47180a482f6d36f05d"
server_key_iv: "5c8a983f88a3f8ffb0aa1033f1cc24e1"
server_key_version: "encryptionkey@0"
I20260806 18:38:37.232095 10404 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786041246094603-20232-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786041246094603-20232-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786041246094603-20232-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 18:38:37.241170 10404 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 18:38:37.244306 10404 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 18:38:37.244504 10404 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 18:38:37.251711 10404 rpc_server.cc:307] RPC server started. Bound to: 127.19.194.62:42545
I20260806 18:38:37.251727 10475 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.194.62:42545 every 8 connection(s)
I20260806 18:38:37.252583 10404 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786041246094603-20232-0/minicluster-data/master-0/data/info.pb
I20260806 18:38:37.255242 10476 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 18:38:37.259935 20232 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJW69tn/build/debug/bin/kudu as pid 10404
I20260806 18:38:37.260084 20232 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786041246094603-20232-0/minicluster-data/master-0/wal/instance
I20260806 18:38:37.260352 20232 external_mini_cluster.cc:1428] Setting key c76b78ce4659576d32206205471cda77
I20260806 18:38:37.260438 10476 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ec530270436c4ef4bb785ecc40919d71: Bootstrap starting.
I20260806 18:38:37.263021 10476 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P ec530270436c4ef4bb785ecc40919d71: Neither blocks nor log segments found. Creating new log.
I20260806 18:38:37.263896 10476 log.cc:824] T 00000000000000000000000000000000 P ec530270436c4ef4bb785ecc40919d71: Log is configured to *not* fsync() on all Append() calls
I20260806 18:38:37.265981 10476 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ec530270436c4ef4bb785ecc40919d71: No bootstrap required, opened a new log
Aug 06 18:38:37 dist-test-slave-1xrh krb5kdc[10390](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041517, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.19.194.62@KRBTEST.COM
I20260806 18:38:37.269790 10476 raft_consensus.cc:348] T 00000000000000000000000000000000 P ec530270436c4ef4bb785ecc40919d71 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ec530270436c4ef4bb785ecc40919d71" member_type: VOTER last_known_addr { host: "127.19.194.62" port: 42545 } }
I20260806 18:38:37.270071 10476 raft_consensus.cc:374] T 00000000000000000000000000000000 P ec530270436c4ef4bb785ecc40919d71 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 18:38:37.270136 10476 raft_consensus.cc:729] T 00000000000000000000000000000000 P ec530270436c4ef4bb785ecc40919d71 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ec530270436c4ef4bb785ecc40919d71, State: Initialized, Role: FOLLOWER
I20260806 18:38:37.270553 10476 consensus_queue.cc:260] T 00000000000000000000000000000000 P ec530270436c4ef4bb785ecc40919d71 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ec530270436c4ef4bb785ecc40919d71" member_type: VOTER last_known_addr { host: "127.19.194.62" port: 42545 } }
I20260806 18:38:37.270668 10476 raft_consensus.cc:388] T 00000000000000000000000000000000 P ec530270436c4ef4bb785ecc40919d71 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 18:38:37.270761 10476 raft_consensus.cc:482] T 00000000000000000000000000000000 P ec530270436c4ef4bb785ecc40919d71 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 18:38:37.270859 10476 raft_consensus.cc:3037] T 00000000000000000000000000000000 P ec530270436c4ef4bb785ecc40919d71 [term 0 FOLLOWER]: Advancing to term 1
I20260806 18:38:37.271871 10476 raft_consensus.cc:504] T 00000000000000000000000000000000 P ec530270436c4ef4bb785ecc40919d71 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ec530270436c4ef4bb785ecc40919d71" member_type: VOTER last_known_addr { host: "127.19.194.62" port: 42545 } }
I20260806 18:38:37.272182 10476 leader_election.cc:302] T 00000000000000000000000000000000 P ec530270436c4ef4bb785ecc40919d71 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: ec530270436c4ef4bb785ecc40919d71; no voters:
I20260806 18:38:37.272527 10476 leader_election.cc:288] T 00000000000000000000000000000000 P ec530270436c4ef4bb785ecc40919d71 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 18:38:37.272615 10481 raft_consensus.cc:2781] T 00000000000000000000000000000000 P ec530270436c4ef4bb785ecc40919d71 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 18:38:37.272877 10481 raft_consensus.cc:686] T 00000000000000000000000000000000 P ec530270436c4ef4bb785ecc40919d71 [term 1 LEADER]: Becoming Leader. State: Replica: ec530270436c4ef4bb785ecc40919d71, State: Running, Role: LEADER
I20260806 18:38:37.273609 10481 consensus_queue.cc:237] T 00000000000000000000000000000000 P ec530270436c4ef4bb785ecc40919d71 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ec530270436c4ef4bb785ecc40919d71" member_type: VOTER last_known_addr { host: "127.19.194.62" port: 42545 } }
I20260806 18:38:37.273792 10476 sys_catalog.cc:564] T 00000000000000000000000000000000 P ec530270436c4ef4bb785ecc40919d71 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 18:38:37.275275 10483 sys_catalog.cc:455] T 00000000000000000000000000000000 P ec530270436c4ef4bb785ecc40919d71 [sys.catalog]: SysCatalogTable state changed. Reason: New leader ec530270436c4ef4bb785ecc40919d71. Latest consensus state: current_term: 1 leader_uuid: "ec530270436c4ef4bb785ecc40919d71" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ec530270436c4ef4bb785ecc40919d71" member_type: VOTER last_known_addr { host: "127.19.194.62" port: 42545 } } }
I20260806 18:38:37.275398 10483 sys_catalog.cc:458] T 00000000000000000000000000000000 P ec530270436c4ef4bb785ecc40919d71 [sys.catalog]: This master's current role is: LEADER
I20260806 18:38:37.275560 10479 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:38:37.261699 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60831 (local address 127.19.194.62:42545)
0806 18:38:37.262106 (+ 407us) server_negotiation.cc:206] Beginning negotiation
0806 18:38:37.262125 (+ 19us) server_negotiation.cc:399] Waiting for connection header
0806 18:38:37.262166 (+ 41us) server_negotiation.cc:407] Connection header received
0806 18:38:37.262970 (+ 804us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:38:37.262994 (+ 24us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:38:37.263439 (+ 445us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:38:37.263794 (+ 355us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:38:37.264607 (+ 813us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:38:37.266000 (+ 1393us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:38:37.266865 (+ 865us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:38:37.267572 (+ 707us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:38:37.267733 (+ 161us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:38:37.270401 (+ 2668us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:38:37.270408 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:38:37.270422 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 18:38:37.270459 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:38:37.272244 (+ 1785us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:38:37.272852 (+ 608us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:38:37.272860 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:38:37.272867 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:38:37.273032 (+ 165us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:38:37.273430 (+ 398us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:38:37.273433 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:38:37.273435 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:38:37.273902 (+ 467us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:38:37.274125 (+ 223us) server_negotiation.cc:1035] Waiting for connection context
0806 18:38:37.274431 (+ 306us) server_negotiation.cc:299] Negotiation successful
0806 18:38:37.274637 (+ 206us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":270,"thread_start_us":105,"threads_started":1}
I20260806 18:38:37.275949 10482 sys_catalog.cc:455] T 00000000000000000000000000000000 P ec530270436c4ef4bb785ecc40919d71 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "ec530270436c4ef4bb785ecc40919d71" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ec530270436c4ef4bb785ecc40919d71" member_type: VOTER last_known_addr { host: "127.19.194.62" port: 42545 } } }
I20260806 18:38:37.276399 10492 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 18:38:37.276428 10482 sys_catalog.cc:458] T 00000000000000000000000000000000 P ec530270436c4ef4bb785ecc40919d71 [sys.catalog]: This master's current role is: LEADER
I20260806 18:38:37.278594 20232 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260806 18:38:37.279346 10492 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 18:38:37.284842 10492 catalog_manager.cc:1347] Generated new cluster ID: 834886286bc140b29f1008ea0391fe58
I20260806 18:38:37.284946 10492 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 18:38:37.289284 10479 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:38:37.280826 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37071 (local address 127.19.194.62:42545)
0806 18:38:37.280940 (+ 114us) server_negotiation.cc:206] Beginning negotiation
0806 18:38:37.280945 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 18:38:37.281180 (+ 235us) server_negotiation.cc:407] Connection header received
0806 18:38:37.281355 (+ 175us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:38:37.281358 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:38:37.281401 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:38:37.281493 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:38:37.282319 (+ 826us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:38:37.282973 (+ 654us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:38:37.283892 (+ 919us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:38:37.284144 (+ 252us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:38:37.284250 (+ 106us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:38:37.285866 (+ 1616us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:38:37.285870 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:38:37.285873 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 18:38:37.285905 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:38:37.287495 (+ 1590us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:38:37.287995 (+ 500us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:38:37.287999 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:38:37.288001 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:38:37.288105 (+ 104us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:38:37.288425 (+ 320us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:38:37.288428 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:38:37.288430 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:38:37.288636 (+ 206us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:38:37.288775 (+ 139us) server_negotiation.cc:1035] Waiting for connection context
0806 18:38:37.289044 (+ 269us) server_negotiation.cc:299] Negotiation successful
0806 18:38:37.289138 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260806 18:38:37.295130 10492 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 18:38:37.296059 10492 catalog_manager.cc:1504] Loading token signing keys...
I20260806 18:38:37.305369 10492 catalog_manager.cc:5868] T 00000000000000000000000000000000 P ec530270436c4ef4bb785ecc40919d71: Generated new TSK 0
I20260806 18:38:37.305940 10492 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 18:38:37.315773 20232 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJW69tn/build/debug/bin/kudu with pid 10404
W20260806 18:38:37.323478 10388 connection.cc:535] client connection to 127.19.194.62:42545 recv error: Network error: failed to read from TLS socket (remote: 127.19.194.62:42545): Connection reset by peer (error 104)
2026-08-06T18:38:37Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/12 (1456 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/13
Loading random data
Initializing database '/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786041246094603-20232-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 18:38:37 dist-test-slave-1xrh krb5kdc[10507](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 18:38:37 dist-test-slave-1xrh krb5kdc[10507](info): set up 2 sockets
Aug 06 18:38:37 dist-test-slave-1xrh krb5kdc[10507](info): commencing operation
krb5kdc: starting...
W20260806 18:38:38.368983 20232 mini_kdc.cc:121] Time spent starting KDC: real 1.031s user 0.006s sys 0.000s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786041246094603-20232-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786041246094603-20232-0/krb5kdc/test-user.keytab.
Aug 06 18:38:38 dist-test-slave-1xrh krb5kdc[10507](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041518, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T18:38:38Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T18:38:38Z Disabled control of system clock
WARNING: no policy specified for kudu/127.19.194.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.19.194.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.19.194.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786041246094603-20232-0/krb5kdc/kudu_127.19.194.62.keytab.
Entry for principal kudu/127.19.194.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786041246094603-20232-0/krb5kdc/kudu_127.19.194.62.keytab.
I20260806 18:38:38.490113 20232 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJW69tn/build/debug/bin/kudu
/tmp/dist-test-taskJW69tn/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786041246094603-20232-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786041246094603-20232-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786041246094603-20232-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786041246094603-20232-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.19.194.62:41655
--webserver_interface=127.19.194.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786041246094603-20232-0/krb5kdc/kudu_127.19.194.62.keytab
--principal=kudu/127.19.194.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.19.194.20:35607
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.19.194.62:41655
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786041246094603-20232-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786041246094603-20232-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786041246094603-20232-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 18:38:38.595747 10520 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 18:38:38.595976 10520 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 18:38:38.596033 10520 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 18:38:38.599855 10520 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 18:38:38.599905 10520 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 18:38:38.599923 10520 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 18:38:38.599938 10520 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 18:38:38.599953 10520 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 18:38:38.605261 10520 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.194.20:35607
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786041246094603-20232-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786041246094603-20232-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.19.194.62:41655
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786041246094603-20232-0/krb5kdc/kudu_127.19.194.62.keytab
--principal=kudu/127.19.194.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.19.194.62:41655
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786041246094603-20232-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.19.194.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10520
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786041246094603-20232-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 17:59:13 UTC on ccycloud.quasar-wvsnzx.root.comops.site
build id 4571028
I20260806 18:38:38.606172 10520 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 18:38:38.607107 10520 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 18:38:38.615617 10525 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:38:38.615617 10526 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:38:38.615654 10529 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 18:38:38.615850 10520 server_base.cc:1034] running on GCE node
I20260806 18:38:38.616385 10520 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 18:38:38.617311 10520 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 18:38:38.618472 10520 hybrid_clock.cc:648] HybridClock initialized: now 1786041518618448 us; error 36 us; skew 500 ppm
Aug 06 18:38:38 dist-test-slave-1xrh krb5kdc[10507](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041518, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.194.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 18:38:38.621222 10520 init.cc:377] Logged in from keytab as kudu/127.19.194.62@KRBTEST.COM (short username kudu)
I20260806 18:38:38.622076 10520 webserver.cc:460] Webserver started at http://127.19.194.62:36641/ using document root <none> and password file <none>
I20260806 18:38:38.622603 10520 fs_manager.cc:359] Metadata directory not provided
I20260806 18:38:38.622689 10520 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 18:38:38.622925 10520 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 18:38:38.624903 10520 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786041246094603-20232-0/minicluster-data/master-0/data/instance:
uuid: "e04397f9a37e4581afc5bfb4c37063c8"
format_stamp: "Formatted at 2026-08-06 18:38:38 on dist-test-slave-1xrh"
server_key: "746efceaf26b84ff8f54017f6179c648"
server_key_iv: "b10d4b3b3bc2990743a5f5678fd3d423"
server_key_version: "encryptionkey@0"
I20260806 18:38:38.625468 10520 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786041246094603-20232-0/minicluster-data/master-0/wal/instance:
uuid: "e04397f9a37e4581afc5bfb4c37063c8"
format_stamp: "Formatted at 2026-08-06 18:38:38 on dist-test-slave-1xrh"
server_key: "746efceaf26b84ff8f54017f6179c648"
server_key_iv: "b10d4b3b3bc2990743a5f5678fd3d423"
server_key_version: "encryptionkey@0"
I20260806 18:38:38.628961 10520 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.005s sys 0.000s
I20260806 18:38:38.631217 10537 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 18:38:38.632318 10520 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260806 18:38:38.632462 10520 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786041246094603-20232-0/minicluster-data/master-0/data,/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786041246094603-20232-0/minicluster-data/master-0/wal
uuid: "e04397f9a37e4581afc5bfb4c37063c8"
format_stamp: "Formatted at 2026-08-06 18:38:38 on dist-test-slave-1xrh"
server_key: "746efceaf26b84ff8f54017f6179c648"
server_key_iv: "b10d4b3b3bc2990743a5f5678fd3d423"
server_key_version: "encryptionkey@0"
I20260806 18:38:38.632568 10520 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786041246094603-20232-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786041246094603-20232-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786041246094603-20232-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 18:38:38.641135 10520 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 18:38:38.644132 10520 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 18:38:38.644325 10520 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 18:38:38.651357 10520 rpc_server.cc:307] RPC server started. Bound to: 127.19.194.62:41655
I20260806 18:38:38.651383 10590 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.194.62:41655 every 8 connection(s)
I20260806 18:38:38.652194 10520 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786041246094603-20232-0/minicluster-data/master-0/data/info.pb
I20260806 18:38:38.654644 10591 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 18:38:38.655599 20232 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJW69tn/build/debug/bin/kudu as pid 10520
I20260806 18:38:38.655728 20232 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786041246094603-20232-0/minicluster-data/master-0/wal/instance
I20260806 18:38:38.655979 20232 external_mini_cluster.cc:1428] Setting key 5e44d6c0d841aed5a57e2b554b53ec62
I20260806 18:38:38.661029 10591 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e04397f9a37e4581afc5bfb4c37063c8: Bootstrap starting.
I20260806 18:38:38.663466 10591 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P e04397f9a37e4581afc5bfb4c37063c8: Neither blocks nor log segments found. Creating new log.
I20260806 18:38:38.664371 10591 log.cc:824] T 00000000000000000000000000000000 P e04397f9a37e4581afc5bfb4c37063c8: Log is configured to *not* fsync() on all Append() calls
I20260806 18:38:38.665657 10594 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:38:38.657219 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50051 (local address 127.19.194.62:41655)
0806 18:38:38.657721 (+ 502us) server_negotiation.cc:206] Beginning negotiation
0806 18:38:38.657743 (+ 22us) server_negotiation.cc:399] Waiting for connection header
0806 18:38:38.657786 (+ 43us) server_negotiation.cc:407] Connection header received
0806 18:38:38.658573 (+ 787us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:38:38.658597 (+ 24us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:38:38.659025 (+ 428us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:38:38.659391 (+ 366us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:38:38.659780 (+ 389us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:38:38.661213 (+ 1433us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:38:38.662116 (+ 903us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:38:38.662777 (+ 661us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:38:38.662946 (+ 169us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:38:38.663800 (+ 854us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:38:38.663806 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:38:38.663822 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 18:38:38.664294 (+ 472us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:38:38.664360 (+ 66us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 18:38:38.664393 (+ 33us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:38:38.664541 (+ 148us) server_negotiation.cc:1035] Waiting for connection context
0806 18:38:38.664736 (+ 195us) server_negotiation.cc:299] Negotiation successful
0806 18:38:38.664864 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":307,"thread_start_us":125,"threads_started":1}
I20260806 18:38:38.666437 10591 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e04397f9a37e4581afc5bfb4c37063c8: No bootstrap required, opened a new log
I20260806 18:38:38.668416 20232 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260806 18:38:38.670251 10591 raft_consensus.cc:348] T 00000000000000000000000000000000 P e04397f9a37e4581afc5bfb4c37063c8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e04397f9a37e4581afc5bfb4c37063c8" member_type: VOTER last_known_addr { host: "127.19.194.62" port: 41655 } }
I20260806 18:38:38.670564 10591 raft_consensus.cc:374] T 00000000000000000000000000000000 P e04397f9a37e4581afc5bfb4c37063c8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 18:38:38.670630 10591 raft_consensus.cc:729] T 00000000000000000000000000000000 P e04397f9a37e4581afc5bfb4c37063c8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e04397f9a37e4581afc5bfb4c37063c8, State: Initialized, Role: FOLLOWER
I20260806 18:38:38.671106 10591 consensus_queue.cc:260] T 00000000000000000000000000000000 P e04397f9a37e4581afc5bfb4c37063c8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e04397f9a37e4581afc5bfb4c37063c8" member_type: VOTER last_known_addr { host: "127.19.194.62" port: 41655 } }
I20260806 18:38:38.671290 10591 raft_consensus.cc:388] T 00000000000000000000000000000000 P e04397f9a37e4581afc5bfb4c37063c8 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 18:38:38.671379 10591 raft_consensus.cc:482] T 00000000000000000000000000000000 P e04397f9a37e4581afc5bfb4c37063c8 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 18:38:38.671504 10591 raft_consensus.cc:3037] T 00000000000000000000000000000000 P e04397f9a37e4581afc5bfb4c37063c8 [term 0 FOLLOWER]: Advancing to term 1
I20260806 18:38:38.672524 10591 raft_consensus.cc:504] T 00000000000000000000000000000000 P e04397f9a37e4581afc5bfb4c37063c8 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e04397f9a37e4581afc5bfb4c37063c8" member_type: VOTER last_known_addr { host: "127.19.194.62" port: 41655 } }
I20260806 18:38:38.672865 10591 leader_election.cc:302] T 00000000000000000000000000000000 P e04397f9a37e4581afc5bfb4c37063c8 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: e04397f9a37e4581afc5bfb4c37063c8; no voters:
I20260806 18:38:38.673224 10591 leader_election.cc:288] T 00000000000000000000000000000000 P e04397f9a37e4581afc5bfb4c37063c8 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 18:38:38.673298 10601 raft_consensus.cc:2781] T 00000000000000000000000000000000 P e04397f9a37e4581afc5bfb4c37063c8 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 18:38:38.673498 10601 raft_consensus.cc:686] T 00000000000000000000000000000000 P e04397f9a37e4581afc5bfb4c37063c8 [term 1 LEADER]: Becoming Leader. State: Replica: e04397f9a37e4581afc5bfb4c37063c8, State: Running, Role: LEADER
I20260806 18:38:38.673815 10601 consensus_queue.cc:237] T 00000000000000000000000000000000 P e04397f9a37e4581afc5bfb4c37063c8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e04397f9a37e4581afc5bfb4c37063c8" member_type: VOTER last_known_addr { host: "127.19.194.62" port: 41655 } }
I20260806 18:38:38.674446 10591 sys_catalog.cc:564] T 00000000000000000000000000000000 P e04397f9a37e4581afc5bfb4c37063c8 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 18:38:38.674790 10594 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:38:38.670341 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60231 (local address 127.19.194.62:41655)
0806 18:38:38.670452 (+ 111us) server_negotiation.cc:206] Beginning negotiation
0806 18:38:38.670455 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 18:38:38.670729 (+ 274us) server_negotiation.cc:407] Connection header received
0806 18:38:38.670887 (+ 158us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:38:38.670890 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:38:38.670935 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:38:38.671013 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:38:38.671385 (+ 372us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:38:38.671952 (+ 567us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:38:38.673073 (+ 1121us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:38:38.673352 (+ 279us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:38:38.673458 (+ 106us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:38:38.674287 (+ 829us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:38:38.674290 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:38:38.674293 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 18:38:38.674304 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:38:38.674334 (+ 30us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 18:38:38.674344 (+ 10us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:38:38.674442 (+ 98us) server_negotiation.cc:1035] Waiting for connection context
0806 18:38:38.674608 (+ 166us) server_negotiation.cc:299] Negotiation successful
0806 18:38:38.674637 (+ 29us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260806 18:38:38.674965 10601 sys_catalog.cc:455] T 00000000000000000000000000000000 P e04397f9a37e4581afc5bfb4c37063c8 [sys.catalog]: SysCatalogTable state changed. Reason: New leader e04397f9a37e4581afc5bfb4c37063c8. Latest consensus state: current_term: 1 leader_uuid: "e04397f9a37e4581afc5bfb4c37063c8" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e04397f9a37e4581afc5bfb4c37063c8" member_type: VOTER last_known_addr { host: "127.19.194.62" port: 41655 } } }
I20260806 18:38:38.675060 10601 sys_catalog.cc:458] T 00000000000000000000000000000000 P e04397f9a37e4581afc5bfb4c37063c8 [sys.catalog]: This master's current role is: LEADER
I20260806 18:38:38.676465 10602 sys_catalog.cc:455] T 00000000000000000000000000000000 P e04397f9a37e4581afc5bfb4c37063c8 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "e04397f9a37e4581afc5bfb4c37063c8" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e04397f9a37e4581afc5bfb4c37063c8" member_type: VOTER last_known_addr { host: "127.19.194.62" port: 41655 } } }
I20260806 18:38:38.676582 10602 sys_catalog.cc:458] T 00000000000000000000000000000000 P e04397f9a37e4581afc5bfb4c37063c8 [sys.catalog]: This master's current role is: LEADER
W20260806 18:38:38.678982 10617 catalog_manager.cc:1558] T 00000000000000000000000000000000 P e04397f9a37e4581afc5bfb4c37063c8: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260806 18:38:38.679071 10617 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260806 18:38:38.679168 10618 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 18:38:38.679858 10618 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 18:38:38.685003 10618 catalog_manager.cc:1347] Generated new cluster ID: 5b858e4829464f37a654f22cb913d9db
I20260806 18:38:38.685062 10618 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 18:38:38.702253 10618 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 18:38:38.703487 10618 catalog_manager.cc:1504] Loading token signing keys...
I20260806 18:38:38.712347 10618 catalog_manager.cc:5868] T 00000000000000000000000000000000 P e04397f9a37e4581afc5bfb4c37063c8: Generated new TSK 0
I20260806 18:38:38.713088 10618 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 18:38:38.720085 20232 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJW69tn/build/debug/bin/kudu with pid 10520
W20260806 18:38:38.727550 10505 connection.cc:535] client connection to 127.19.194.62:41655 recv error: Network error: failed to read from TLS socket (remote: 127.19.194.62:41655): Connection reset by peer (error 104)
2026-08-06T18:38:38Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/13 (1403 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/14
Loading random data
Initializing database '/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786041246094603-20232-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 18:38:38 dist-test-slave-1xrh krb5kdc[10625](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 18:38:38 dist-test-slave-1xrh krb5kdc[10625](info): set up 2 sockets
Aug 06 18:38:38 dist-test-slave-1xrh krb5kdc[10625](info): commencing operation
krb5kdc: starting...
W20260806 18:38:39.760360 20232 mini_kdc.cc:121] Time spent starting KDC: real 1.019s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786041246094603-20232-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786041246094603-20232-0/krb5kdc/test-user.keytab.
Aug 06 18:38:39 dist-test-slave-1xrh krb5kdc[10625](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041519, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T18:38:39Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T18:38:39Z Disabled control of system clock
WARNING: no policy specified for kudu/127.19.194.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.19.194.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.19.194.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786041246094603-20232-0/krb5kdc/kudu_127.19.194.62.keytab.
Entry for principal kudu/127.19.194.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786041246094603-20232-0/krb5kdc/kudu_127.19.194.62.keytab.
I20260806 18:38:39.880620 20232 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJW69tn/build/debug/bin/kudu
/tmp/dist-test-taskJW69tn/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786041246094603-20232-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786041246094603-20232-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786041246094603-20232-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786041246094603-20232-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.19.194.62:34933
--webserver_interface=127.19.194.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786041246094603-20232-0/krb5kdc/kudu_127.19.194.62.keytab
--principal=kudu/127.19.194.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.19.194.20:35565
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.19.194.62:34933
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786041246094603-20232-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786041246094603-20232-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786041246094603-20232-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 18:38:39.990409 10639 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 18:38:39.990693 10639 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 18:38:39.990737 10639 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 18:38:39.994846 10639 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 18:38:39.994926 10639 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 18:38:39.994946 10639 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 18:38:39.994963 10639 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 18:38:39.995023 10639 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 18:38:40.000481 10639 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.194.20:35565
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786041246094603-20232-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786041246094603-20232-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.19.194.62:34933
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786041246094603-20232-0/krb5kdc/kudu_127.19.194.62.keytab
--principal=kudu/127.19.194.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.19.194.62:34933
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786041246094603-20232-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.19.194.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10639
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786041246094603-20232-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 17:59:13 UTC on ccycloud.quasar-wvsnzx.root.comops.site
build id 4571028
I20260806 18:38:40.001647 10639 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 18:38:40.002753 10639 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 18:38:40.011804 10647 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:38:40.011823 10645 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:38:40.011822 10644 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 18:38:40.011876 10639 server_base.cc:1034] running on GCE node
I20260806 18:38:40.012634 10639 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 18:38:40.013759 10639 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 18:38:40.014945 10639 hybrid_clock.cc:648] HybridClock initialized: now 1786041520014915 us; error 41 us; skew 500 ppm
Aug 06 18:38:40 dist-test-slave-1xrh krb5kdc[10625](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041520, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.194.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 18:38:40.018262 10639 init.cc:377] Logged in from keytab as kudu/127.19.194.62@KRBTEST.COM (short username kudu)
I20260806 18:38:40.019253 10639 webserver.cc:460] Webserver started at http://127.19.194.62:36567/ using document root <none> and password file <none>
I20260806 18:38:40.019814 10639 fs_manager.cc:359] Metadata directory not provided
I20260806 18:38:40.019876 10639 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 18:38:40.020123 10639 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 18:38:40.022186 10639 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786041246094603-20232-0/minicluster-data/master-0/data/instance:
uuid: "7a47896624954b0689ac42791b98e2c2"
format_stamp: "Formatted at 2026-08-06 18:38:40 on dist-test-slave-1xrh"
server_key: "7abae3a550766d83622fccb51b6f92d9"
server_key_iv: "5324cbaf833d0ae1181d3d74ed39a4ae"
server_key_version: "encryptionkey@0"
I20260806 18:38:40.022761 10639 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786041246094603-20232-0/minicluster-data/master-0/wal/instance:
uuid: "7a47896624954b0689ac42791b98e2c2"
format_stamp: "Formatted at 2026-08-06 18:38:40 on dist-test-slave-1xrh"
server_key: "7abae3a550766d83622fccb51b6f92d9"
server_key_iv: "5324cbaf833d0ae1181d3d74ed39a4ae"
server_key_version: "encryptionkey@0"
I20260806 18:38:40.026522 10639 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.003s
I20260806 18:38:40.028981 10655 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 18:38:40.030217 10639 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260806 18:38:40.030350 10639 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786041246094603-20232-0/minicluster-data/master-0/data,/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786041246094603-20232-0/minicluster-data/master-0/wal
uuid: "7a47896624954b0689ac42791b98e2c2"
format_stamp: "Formatted at 2026-08-06 18:38:40 on dist-test-slave-1xrh"
server_key: "7abae3a550766d83622fccb51b6f92d9"
server_key_iv: "5324cbaf833d0ae1181d3d74ed39a4ae"
server_key_version: "encryptionkey@0"
I20260806 18:38:40.030463 10639 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786041246094603-20232-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786041246094603-20232-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786041246094603-20232-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 18:38:40.035215 10639 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 18:38:40.038471 10639 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 18:38:40.038676 10639 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 18:38:40.045576 10639 rpc_server.cc:307] RPC server started. Bound to: 127.19.194.62:34933
I20260806 18:38:40.045589 10719 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.194.62:34933 every 8 connection(s)
I20260806 18:38:40.046499 10639 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786041246094603-20232-0/minicluster-data/master-0/data/info.pb
I20260806 18:38:40.049202 10720 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 18:38:40.054370 10720 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7a47896624954b0689ac42791b98e2c2: Bootstrap starting.
I20260806 18:38:40.056730 20232 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJW69tn/build/debug/bin/kudu as pid 10639
I20260806 18:38:40.056802 10720 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 7a47896624954b0689ac42791b98e2c2: Neither blocks nor log segments found. Creating new log.
I20260806 18:38:40.056880 20232 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786041246094603-20232-0/minicluster-data/master-0/wal/instance
I20260806 18:38:40.057221 20232 external_mini_cluster.cc:1428] Setting key 5090c98f7a5c47a94805e69f3145b8f3
I20260806 18:38:40.057667 10720 log.cc:824] T 00000000000000000000000000000000 P 7a47896624954b0689ac42791b98e2c2: Log is configured to *not* fsync() on all Append() calls
I20260806 18:38:40.059677 10720 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7a47896624954b0689ac42791b98e2c2: No bootstrap required, opened a new log
I20260806 18:38:40.062237 10725 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:38:40.058369 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51971 (local address 127.19.194.62:34933)
0806 18:38:40.058811 (+ 442us) server_negotiation.cc:206] Beginning negotiation
0806 18:38:40.058825 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0806 18:38:40.058861 (+ 36us) server_negotiation.cc:407] Connection header received
0806 18:38:40.059522 (+ 661us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:38:40.059540 (+ 18us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:38:40.059836 (+ 296us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:38:40.060125 (+ 289us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:38:40.061051 (+ 926us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:38:40.061056 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:38:40.061068 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 18:38:40.061079 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:38:40.061137 (+ 58us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 18:38:40.061155 (+ 18us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:38:40.061243 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0806 18:38:40.061362 (+ 119us) server_negotiation.cc:299] Negotiation successful
0806 18:38:40.061524 (+ 162us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":274,"thread_start_us":124,"threads_started":1}
I20260806 18:38:40.063516 10720 raft_consensus.cc:348] T 00000000000000000000000000000000 P 7a47896624954b0689ac42791b98e2c2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7a47896624954b0689ac42791b98e2c2" member_type: VOTER last_known_addr { host: "127.19.194.62" port: 34933 } }
I20260806 18:38:40.063813 10720 raft_consensus.cc:374] T 00000000000000000000000000000000 P 7a47896624954b0689ac42791b98e2c2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 18:38:40.063897 10720 raft_consensus.cc:729] T 00000000000000000000000000000000 P 7a47896624954b0689ac42791b98e2c2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7a47896624954b0689ac42791b98e2c2, State: Initialized, Role: FOLLOWER
I20260806 18:38:40.064352 10720 consensus_queue.cc:260] T 00000000000000000000000000000000 P 7a47896624954b0689ac42791b98e2c2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7a47896624954b0689ac42791b98e2c2" member_type: VOTER last_known_addr { host: "127.19.194.62" port: 34933 } }
I20260806 18:38:40.064550 10720 raft_consensus.cc:388] T 00000000000000000000000000000000 P 7a47896624954b0689ac42791b98e2c2 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 18:38:40.064621 10720 raft_consensus.cc:482] T 00000000000000000000000000000000 P 7a47896624954b0689ac42791b98e2c2 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 18:38:40.064713 10720 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 7a47896624954b0689ac42791b98e2c2 [term 0 FOLLOWER]: Advancing to term 1
I20260806 18:38:40.065627 20232 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260806 18:38:40.065804 10720 raft_consensus.cc:504] T 00000000000000000000000000000000 P 7a47896624954b0689ac42791b98e2c2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7a47896624954b0689ac42791b98e2c2" member_type: VOTER last_known_addr { host: "127.19.194.62" port: 34933 } }
I20260806 18:38:40.066126 10720 leader_election.cc:302] T 00000000000000000000000000000000 P 7a47896624954b0689ac42791b98e2c2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 7a47896624954b0689ac42791b98e2c2; no voters:
I20260806 18:38:40.066426 10720 leader_election.cc:288] T 00000000000000000000000000000000 P 7a47896624954b0689ac42791b98e2c2 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 18:38:40.066538 10726 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 7a47896624954b0689ac42791b98e2c2 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 18:38:40.066748 10726 raft_consensus.cc:686] T 00000000000000000000000000000000 P 7a47896624954b0689ac42791b98e2c2 [term 1 LEADER]: Becoming Leader. State: Replica: 7a47896624954b0689ac42791b98e2c2, State: Running, Role: LEADER
I20260806 18:38:40.067171 10726 consensus_queue.cc:237] T 00000000000000000000000000000000 P 7a47896624954b0689ac42791b98e2c2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7a47896624954b0689ac42791b98e2c2" member_type: VOTER last_known_addr { host: "127.19.194.62" port: 34933 } }
I20260806 18:38:40.067662 10720 sys_catalog.cc:564] T 00000000000000000000000000000000 P 7a47896624954b0689ac42791b98e2c2 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 18:38:40.068418 10729 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7a47896624954b0689ac42791b98e2c2 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 7a47896624954b0689ac42791b98e2c2. Latest consensus state: current_term: 1 leader_uuid: "7a47896624954b0689ac42791b98e2c2" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7a47896624954b0689ac42791b98e2c2" member_type: VOTER last_known_addr { host: "127.19.194.62" port: 34933 } } }
I20260806 18:38:40.068645 10729 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7a47896624954b0689ac42791b98e2c2 [sys.catalog]: This master's current role is: LEADER
I20260806 18:38:40.068792 10727 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7a47896624954b0689ac42791b98e2c2 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "7a47896624954b0689ac42791b98e2c2" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7a47896624954b0689ac42791b98e2c2" member_type: VOTER last_known_addr { host: "127.19.194.62" port: 34933 } } }
I20260806 18:38:40.068958 10727 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7a47896624954b0689ac42791b98e2c2 [sys.catalog]: This master's current role is: LEADER
I20260806 18:38:40.069113 10741 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 18:38:40.070166 10725 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:38:40.067920 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51785 (local address 127.19.194.62:34933)
0806 18:38:40.068020 (+ 100us) server_negotiation.cc:206] Beginning negotiation
0806 18:38:40.068024 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 18:38:40.068427 (+ 403us) server_negotiation.cc:407] Connection header received
0806 18:38:40.068642 (+ 215us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:38:40.068648 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:38:40.068687 (+ 39us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:38:40.068743 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:38:40.069628 (+ 885us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:38:40.069634 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:38:40.069641 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 18:38:40.069654 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:38:40.069682 (+ 28us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 18:38:40.069700 (+ 18us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:38:40.069766 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0806 18:38:40.069983 (+ 217us) server_negotiation.cc:299] Negotiation successful
0806 18:38:40.070031 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":29}
I20260806 18:38:40.072614 10741 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 18:38:40.078069 10741 catalog_manager.cc:1347] Generated new cluster ID: 540554e44944414f9fa086d4458ab38d
I20260806 18:38:40.078135 10741 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 18:38:40.087410 10741 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 18:38:40.087803 10741 catalog_manager.cc:1504] Loading token signing keys...
I20260806 18:38:40.092015 10741 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 7a47896624954b0689ac42791b98e2c2: Generated new TSK 0
I20260806 18:38:40.092780 10741 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 18:38:40.096974 20232 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJW69tn/build/debug/bin/kudu with pid 10639
2026-08-06T18:38:40Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/14 (1376 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/15
Loading random data
Initializing database '/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786041246094603-20232-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 18:38:40 dist-test-slave-1xrh krb5kdc[10754](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 18:38:40 dist-test-slave-1xrh krb5kdc[10754](info): set up 2 sockets
Aug 06 18:38:40 dist-test-slave-1xrh krb5kdc[10754](info): commencing operation
krb5kdc: starting...
W20260806 18:38:41.232339 20232 mini_kdc.cc:121] Time spent starting KDC: real 1.114s user 0.001s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786041246094603-20232-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786041246094603-20232-0/krb5kdc/test-user.keytab.
Aug 06 18:38:41 dist-test-slave-1xrh krb5kdc[10754](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041521, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T18:38:41Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T18:38:41Z Disabled control of system clock
WARNING: no policy specified for kudu/127.19.194.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.19.194.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.19.194.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786041246094603-20232-0/krb5kdc/kudu_127.19.194.62.keytab.
Entry for principal kudu/127.19.194.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786041246094603-20232-0/krb5kdc/kudu_127.19.194.62.keytab.
I20260806 18:38:41.356309 20232 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJW69tn/build/debug/bin/kudu
/tmp/dist-test-taskJW69tn/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786041246094603-20232-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786041246094603-20232-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786041246094603-20232-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786041246094603-20232-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.19.194.62:44103
--webserver_interface=127.19.194.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786041246094603-20232-0/krb5kdc/kudu_127.19.194.62.keytab
--principal=kudu/127.19.194.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.19.194.20:34715
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.19.194.62:44103
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786041246094603-20232-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786041246094603-20232-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786041246094603-20232-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 18:38:41.465080 10768 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 18:38:41.465366 10768 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 18:38:41.465449 10768 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 18:38:41.469424 10768 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 18:38:41.469508 10768 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 18:38:41.469547 10768 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 18:38:41.469614 10768 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 18:38:41.469652 10768 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 18:38:41.474786 10768 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.194.20:34715
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786041246094603-20232-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786041246094603-20232-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.19.194.62:44103
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786041246094603-20232-0/krb5kdc/kudu_127.19.194.62.keytab
--principal=kudu/127.19.194.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.19.194.62:44103
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786041246094603-20232-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.19.194.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10768
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786041246094603-20232-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 17:59:13 UTC on ccycloud.quasar-wvsnzx.root.comops.site
build id 4571028
I20260806 18:38:41.475868 10768 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 18:38:41.476857 10768 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 18:38:41.485253 10773 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:38:41.485234 10777 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:38:41.485262 10775 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 18:38:41.485522 10768 server_base.cc:1034] running on GCE node
I20260806 18:38:41.486125 10768 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 18:38:41.487208 10768 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 18:38:41.488394 10768 hybrid_clock.cc:648] HybridClock initialized: now 1786041521488380 us; error 29 us; skew 500 ppm
Aug 06 18:38:41 dist-test-slave-1xrh krb5kdc[10754](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041521, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.194.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 18:38:41.498706 10768 init.cc:377] Logged in from keytab as kudu/127.19.194.62@KRBTEST.COM (short username kudu)
I20260806 18:38:41.499689 10768 webserver.cc:460] Webserver started at http://127.19.194.62:46503/ using document root <none> and password file <none>
I20260806 18:38:41.500250 10768 fs_manager.cc:359] Metadata directory not provided
I20260806 18:38:41.500344 10768 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 18:38:41.500586 10768 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 18:38:41.502640 10768 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786041246094603-20232-0/minicluster-data/master-0/data/instance:
uuid: "16d15c131f0b4e01a919399c0e3ba69b"
format_stamp: "Formatted at 2026-08-06 18:38:41 on dist-test-slave-1xrh"
server_key: "4700fdef394acde5736b182e5b1f1c35"
server_key_iv: "3e4081bc2feedd299ce1a9fc7a3fcbb2"
server_key_version: "encryptionkey@0"
I20260806 18:38:41.503293 10768 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786041246094603-20232-0/minicluster-data/master-0/wal/instance:
uuid: "16d15c131f0b4e01a919399c0e3ba69b"
format_stamp: "Formatted at 2026-08-06 18:38:41 on dist-test-slave-1xrh"
server_key: "4700fdef394acde5736b182e5b1f1c35"
server_key_iv: "3e4081bc2feedd299ce1a9fc7a3fcbb2"
server_key_version: "encryptionkey@0"
I20260806 18:38:41.506908 10768 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.000s
I20260806 18:38:41.509234 10785 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 18:38:41.510521 10768 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260806 18:38:41.510706 10768 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786041246094603-20232-0/minicluster-data/master-0/data,/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786041246094603-20232-0/minicluster-data/master-0/wal
uuid: "16d15c131f0b4e01a919399c0e3ba69b"
format_stamp: "Formatted at 2026-08-06 18:38:41 on dist-test-slave-1xrh"
server_key: "4700fdef394acde5736b182e5b1f1c35"
server_key_iv: "3e4081bc2feedd299ce1a9fc7a3fcbb2"
server_key_version: "encryptionkey@0"
I20260806 18:38:41.510834 10768 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786041246094603-20232-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786041246094603-20232-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786041246094603-20232-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 18:38:41.523295 10768 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 18:38:41.526110 10768 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 18:38:41.526328 10768 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 18:38:41.533875 10768 rpc_server.cc:307] RPC server started. Bound to: 127.19.194.62:44103
I20260806 18:38:41.533921 10840 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.194.62:44103 every 8 connection(s)
I20260806 18:38:41.534865 10768 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786041246094603-20232-0/minicluster-data/master-0/data/info.pb
I20260806 18:38:41.537689 10841 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 18:38:41.542253 20232 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJW69tn/build/debug/bin/kudu as pid 10768
I20260806 18:38:41.542415 20232 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786041246094603-20232-0/minicluster-data/master-0/wal/instance
I20260806 18:38:41.542724 20232 external_mini_cluster.cc:1428] Setting key 6d2ad7c51360e7cf594132047135361f
I20260806 18:38:41.543421 10841 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 16d15c131f0b4e01a919399c0e3ba69b: Bootstrap starting.
I20260806 18:38:41.545732 10841 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 16d15c131f0b4e01a919399c0e3ba69b: Neither blocks nor log segments found. Creating new log.
I20260806 18:38:41.546656 10841 log.cc:824] T 00000000000000000000000000000000 P 16d15c131f0b4e01a919399c0e3ba69b: Log is configured to *not* fsync() on all Append() calls
I20260806 18:38:41.548731 10841 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 16d15c131f0b4e01a919399c0e3ba69b: No bootstrap required, opened a new log
Aug 06 18:38:41 dist-test-slave-1xrh krb5kdc[10754](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041521, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.19.194.62@KRBTEST.COM
I20260806 18:38:41.553069 10841 raft_consensus.cc:348] T 00000000000000000000000000000000 P 16d15c131f0b4e01a919399c0e3ba69b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "16d15c131f0b4e01a919399c0e3ba69b" member_type: VOTER last_known_addr { host: "127.19.194.62" port: 44103 } }
I20260806 18:38:41.553364 10841 raft_consensus.cc:374] T 00000000000000000000000000000000 P 16d15c131f0b4e01a919399c0e3ba69b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 18:38:41.553475 10841 raft_consensus.cc:729] T 00000000000000000000000000000000 P 16d15c131f0b4e01a919399c0e3ba69b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 16d15c131f0b4e01a919399c0e3ba69b, State: Initialized, Role: FOLLOWER
I20260806 18:38:41.553915 10841 consensus_queue.cc:260] T 00000000000000000000000000000000 P 16d15c131f0b4e01a919399c0e3ba69b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "16d15c131f0b4e01a919399c0e3ba69b" member_type: VOTER last_known_addr { host: "127.19.194.62" port: 44103 } }
I20260806 18:38:41.554032 10841 raft_consensus.cc:388] T 00000000000000000000000000000000 P 16d15c131f0b4e01a919399c0e3ba69b [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 18:38:41.554116 10841 raft_consensus.cc:482] T 00000000000000000000000000000000 P 16d15c131f0b4e01a919399c0e3ba69b [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 18:38:41.554234 10841 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 16d15c131f0b4e01a919399c0e3ba69b [term 0 FOLLOWER]: Advancing to term 1
I20260806 18:38:41.555049 10841 raft_consensus.cc:504] T 00000000000000000000000000000000 P 16d15c131f0b4e01a919399c0e3ba69b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "16d15c131f0b4e01a919399c0e3ba69b" member_type: VOTER last_known_addr { host: "127.19.194.62" port: 44103 } }
I20260806 18:38:41.555416 10841 leader_election.cc:302] T 00000000000000000000000000000000 P 16d15c131f0b4e01a919399c0e3ba69b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 16d15c131f0b4e01a919399c0e3ba69b; no voters:
I20260806 18:38:41.555742 10841 leader_election.cc:288] T 00000000000000000000000000000000 P 16d15c131f0b4e01a919399c0e3ba69b [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 18:38:41.555806 10846 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 16d15c131f0b4e01a919399c0e3ba69b [term 1 FOLLOWER]: Leader election won for term 1
I20260806 18:38:41.556061 10846 raft_consensus.cc:686] T 00000000000000000000000000000000 P 16d15c131f0b4e01a919399c0e3ba69b [term 1 LEADER]: Becoming Leader. State: Replica: 16d15c131f0b4e01a919399c0e3ba69b, State: Running, Role: LEADER
I20260806 18:38:41.556380 10846 consensus_queue.cc:237] T 00000000000000000000000000000000 P 16d15c131f0b4e01a919399c0e3ba69b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "16d15c131f0b4e01a919399c0e3ba69b" member_type: VOTER last_known_addr { host: "127.19.194.62" port: 44103 } }
I20260806 18:38:41.557055 10841 sys_catalog.cc:564] T 00000000000000000000000000000000 P 16d15c131f0b4e01a919399c0e3ba69b [sys.catalog]: configured and running, proceeding with master startup.
I20260806 18:38:41.557361 10844 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:38:41.544104 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59225 (local address 127.19.194.62:44103)
0806 18:38:41.544498 (+ 394us) server_negotiation.cc:206] Beginning negotiation
0806 18:38:41.544524 (+ 26us) server_negotiation.cc:399] Waiting for connection header
0806 18:38:41.544559 (+ 35us) server_negotiation.cc:407] Connection header received
0806 18:38:41.545155 (+ 596us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:38:41.545171 (+ 16us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:38:41.545491 (+ 320us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:38:41.545776 (+ 285us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:38:41.546710 (+ 934us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:38:41.547766 (+ 1056us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:38:41.548942 (+ 1176us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:38:41.549382 (+ 440us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:38:41.549519 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:38:41.552294 (+ 2775us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:38:41.552299 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:38:41.552311 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 18:38:41.552354 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:38:41.554347 (+ 1993us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:38:41.554801 (+ 454us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:38:41.554807 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:38:41.554813 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:38:41.554936 (+ 123us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:38:41.555266 (+ 330us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:38:41.555269 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:38:41.555271 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:38:41.555720 (+ 449us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:38:41.555951 (+ 231us) server_negotiation.cc:1035] Waiting for connection context
0806 18:38:41.556223 (+ 272us) server_negotiation.cc:299] Negotiation successful
0806 18:38:41.556536 (+ 313us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":247,"thread_start_us":121,"threads_started":1}
I20260806 18:38:41.557574 10846 sys_catalog.cc:455] T 00000000000000000000000000000000 P 16d15c131f0b4e01a919399c0e3ba69b [sys.catalog]: SysCatalogTable state changed. Reason: New leader 16d15c131f0b4e01a919399c0e3ba69b. Latest consensus state: current_term: 1 leader_uuid: "16d15c131f0b4e01a919399c0e3ba69b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "16d15c131f0b4e01a919399c0e3ba69b" member_type: VOTER last_known_addr { host: "127.19.194.62" port: 44103 } } }
I20260806 18:38:41.557710 10846 sys_catalog.cc:458] T 00000000000000000000000000000000 P 16d15c131f0b4e01a919399c0e3ba69b [sys.catalog]: This master's current role is: LEADER
I20260806 18:38:41.557993 10846 sys_catalog.cc:455] T 00000000000000000000000000000000 P 16d15c131f0b4e01a919399c0e3ba69b [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "16d15c131f0b4e01a919399c0e3ba69b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "16d15c131f0b4e01a919399c0e3ba69b" member_type: VOTER last_known_addr { host: "127.19.194.62" port: 44103 } } }
I20260806 18:38:41.558140 10846 sys_catalog.cc:458] T 00000000000000000000000000000000 P 16d15c131f0b4e01a919399c0e3ba69b [sys.catalog]: This master's current role is: LEADER
I20260806 18:38:41.558449 10852 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 18:38:41.560537 20232 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260806 18:38:41.561745 10852 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 18:38:41.567950 10852 catalog_manager.cc:1347] Generated new cluster ID: fca045e98a3a4d2a8fb2f152739abe34
I20260806 18:38:41.568369 10852 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 18:38:41.571200 10844 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:38:41.563101 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40897 (local address 127.19.194.62:44103)
0806 18:38:41.563250 (+ 149us) server_negotiation.cc:206] Beginning negotiation
0806 18:38:41.563256 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 18:38:41.563410 (+ 154us) server_negotiation.cc:407] Connection header received
0806 18:38:41.563536 (+ 126us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:38:41.563539 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:38:41.563583 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:38:41.563660 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:38:41.564499 (+ 839us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:38:41.565008 (+ 509us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:38:41.565861 (+ 853us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:38:41.566196 (+ 335us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:38:41.566259 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:38:41.567772 (+ 1513us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:38:41.567779 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:38:41.567781 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 18:38:41.567811 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:38:41.569538 (+ 1727us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:38:41.570005 (+ 467us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:38:41.570008 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:38:41.570010 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:38:41.570077 (+ 67us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:38:41.570344 (+ 267us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:38:41.570346 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:38:41.570348 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:38:41.570534 (+ 186us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:38:41.570622 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0806 18:38:41.570935 (+ 313us) server_negotiation.cc:299] Negotiation successful
0806 18:38:41.571036 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":76}
I20260806 18:38:41.579394 10852 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 18:38:41.580498 10852 catalog_manager.cc:1504] Loading token signing keys...
I20260806 18:38:41.584909 10852 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 16d15c131f0b4e01a919399c0e3ba69b: Generated new TSK 0
I20260806 18:38:41.585626 10852 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 18:38:41.593374 20232 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJW69tn/build/debug/bin/kudu with pid 10768
2026-08-06T18:38:41Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/15 (1496 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/16
Loading random data
Initializing database '/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786041246094603-20232-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 18:38:41 dist-test-slave-1xrh krb5kdc[10873](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 18:38:41 dist-test-slave-1xrh krb5kdc[10873](info): set up 2 sockets
Aug 06 18:38:41 dist-test-slave-1xrh krb5kdc[10873](info): commencing operation
krb5kdc: starting...
W20260806 18:38:42.652879 20232 mini_kdc.cc:121] Time spent starting KDC: real 1.038s user 0.001s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786041246094603-20232-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786041246094603-20232-0/krb5kdc/test-user.keytab.
Aug 06 18:38:42 dist-test-slave-1xrh krb5kdc[10873](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041522, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T18:38:42Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T18:38:42Z Disabled control of system clock
WARNING: no policy specified for kudu/127.19.194.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.19.194.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.19.194.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786041246094603-20232-0/krb5kdc/kudu_127.19.194.62.keytab.
Entry for principal kudu/127.19.194.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786041246094603-20232-0/krb5kdc/kudu_127.19.194.62.keytab.
I20260806 18:38:42.780200 20232 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJW69tn/build/debug/bin/kudu
/tmp/dist-test-taskJW69tn/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786041246094603-20232-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786041246094603-20232-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786041246094603-20232-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786041246094603-20232-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.19.194.62:37647
--webserver_interface=127.19.194.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786041246094603-20232-0/krb5kdc/kudu_127.19.194.62.keytab
--principal=kudu/127.19.194.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.19.194.20:37863
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.19.194.62:37647
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786041246094603-20232-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786041246094603-20232-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786041246094603-20232-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 18:38:42.892690 10886 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 18:38:42.892948 10886 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 18:38:42.893013 10886 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 18:38:42.897045 10886 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 18:38:42.897101 10886 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 18:38:42.897121 10886 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 18:38:42.897137 10886 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 18:38:42.897195 10886 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 18:38:42.902251 10886 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.194.20:37863
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786041246094603-20232-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786041246094603-20232-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.19.194.62:37647
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786041246094603-20232-0/krb5kdc/kudu_127.19.194.62.keytab
--principal=kudu/127.19.194.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.19.194.62:37647
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786041246094603-20232-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.19.194.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10886
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786041246094603-20232-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 17:59:13 UTC on ccycloud.quasar-wvsnzx.root.comops.site
build id 4571028
I20260806 18:38:42.903288 10886 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 18:38:42.904284 10886 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 18:38:42.912632 10891 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:38:42.912621 10894 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 18:38:42.912762 10886 server_base.cc:1034] running on GCE node
W20260806 18:38:42.912621 10892 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 18:38:42.913352 10886 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 18:38:42.914374 10886 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 18:38:42.915530 10886 hybrid_clock.cc:648] HybridClock initialized: now 1786041522915513 us; error 29 us; skew 500 ppm
Aug 06 18:38:42 dist-test-slave-1xrh krb5kdc[10873](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041522, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.194.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 18:38:42.918390 10886 init.cc:377] Logged in from keytab as kudu/127.19.194.62@KRBTEST.COM (short username kudu)
I20260806 18:38:42.919286 10886 webserver.cc:460] Webserver started at http://127.19.194.62:36735/ using document root <none> and password file <none>
I20260806 18:38:42.919826 10886 fs_manager.cc:359] Metadata directory not provided
I20260806 18:38:42.919943 10886 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 18:38:42.920171 10886 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 18:38:42.922164 10886 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786041246094603-20232-0/minicluster-data/master-0/data/instance:
uuid: "cacda9f469af4273a0e007ddc2c370b1"
format_stamp: "Formatted at 2026-08-06 18:38:42 on dist-test-slave-1xrh"
server_key: "9c300f29a0ea16a1a09c84c842ace0ae"
server_key_iv: "ccfceebc59f8a4361d3540f0fc5cc881"
server_key_version: "encryptionkey@0"
I20260806 18:38:42.922731 10886 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786041246094603-20232-0/minicluster-data/master-0/wal/instance:
uuid: "cacda9f469af4273a0e007ddc2c370b1"
format_stamp: "Formatted at 2026-08-06 18:38:42 on dist-test-slave-1xrh"
server_key: "9c300f29a0ea16a1a09c84c842ace0ae"
server_key_iv: "ccfceebc59f8a4361d3540f0fc5cc881"
server_key_version: "encryptionkey@0"
I20260806 18:38:42.926270 10886 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.006s sys 0.000s
I20260806 18:38:42.928745 10902 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 18:38:42.930011 10886 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260806 18:38:42.930171 10886 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786041246094603-20232-0/minicluster-data/master-0/data,/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786041246094603-20232-0/minicluster-data/master-0/wal
uuid: "cacda9f469af4273a0e007ddc2c370b1"
format_stamp: "Formatted at 2026-08-06 18:38:42 on dist-test-slave-1xrh"
server_key: "9c300f29a0ea16a1a09c84c842ace0ae"
server_key_iv: "ccfceebc59f8a4361d3540f0fc5cc881"
server_key_version: "encryptionkey@0"
I20260806 18:38:42.930284 10886 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786041246094603-20232-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786041246094603-20232-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786041246094603-20232-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 18:38:42.943341 10886 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 18:38:42.946481 10886 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 18:38:42.946681 10886 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 18:38:42.953621 10886 rpc_server.cc:307] RPC server started. Bound to: 127.19.194.62:37647
I20260806 18:38:42.953655 10957 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.194.62:37647 every 8 connection(s)
I20260806 18:38:42.954520 10886 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786041246094603-20232-0/minicluster-data/master-0/data/info.pb
I20260806 18:38:42.956929 20232 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJW69tn/build/debug/bin/kudu as pid 10886
I20260806 18:38:42.957069 20232 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786041246094603-20232-0/minicluster-data/master-0/wal/instance
I20260806 18:38:42.957402 20232 external_mini_cluster.cc:1428] Setting key b61a25038ac03c8b8ab6aee26886ca84
I20260806 18:38:42.957479 10958 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 18:38:42.964434 10958 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P cacda9f469af4273a0e007ddc2c370b1: Bootstrap starting.
I20260806 18:38:42.966847 10961 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:38:42.958795 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48277 (local address 127.19.194.62:37647)
0806 18:38:42.959285 (+ 490us) server_negotiation.cc:206] Beginning negotiation
0806 18:38:42.959301 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0806 18:38:42.959356 (+ 55us) server_negotiation.cc:407] Connection header received
0806 18:38:42.960195 (+ 839us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:38:42.960213 (+ 18us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:38:42.960564 (+ 351us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:38:42.960855 (+ 291us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:38:42.961210 (+ 355us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:38:42.962174 (+ 964us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:38:42.963475 (+ 1301us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:38:42.963969 (+ 494us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:38:42.964099 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:38:42.965055 (+ 956us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:38:42.965060 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:38:42.965072 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 18:38:42.965541 (+ 469us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:38:42.965599 (+ 58us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 18:38:42.965622 (+ 23us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:38:42.965705 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0806 18:38:42.965873 (+ 168us) server_negotiation.cc:299] Negotiation successful
0806 18:38:42.966112 (+ 239us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":334,"thread_start_us":153,"threads_started":1}
I20260806 18:38:42.966872 10958 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P cacda9f469af4273a0e007ddc2c370b1: Neither blocks nor log segments found. Creating new log.
I20260806 18:38:42.967772 10958 log.cc:824] T 00000000000000000000000000000000 P cacda9f469af4273a0e007ddc2c370b1: Log is configured to *not* fsync() on all Append() calls
I20260806 18:38:42.969734 20232 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260806 18:38:42.969898 10958 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P cacda9f469af4273a0e007ddc2c370b1: No bootstrap required, opened a new log
I20260806 18:38:42.973636 10958 raft_consensus.cc:348] T 00000000000000000000000000000000 P cacda9f469af4273a0e007ddc2c370b1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cacda9f469af4273a0e007ddc2c370b1" member_type: VOTER last_known_addr { host: "127.19.194.62" port: 37647 } }
I20260806 18:38:42.973881 10958 raft_consensus.cc:374] T 00000000000000000000000000000000 P cacda9f469af4273a0e007ddc2c370b1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 18:38:42.973981 10958 raft_consensus.cc:729] T 00000000000000000000000000000000 P cacda9f469af4273a0e007ddc2c370b1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cacda9f469af4273a0e007ddc2c370b1, State: Initialized, Role: FOLLOWER
I20260806 18:38:42.974442 10958 consensus_queue.cc:260] T 00000000000000000000000000000000 P cacda9f469af4273a0e007ddc2c370b1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cacda9f469af4273a0e007ddc2c370b1" member_type: VOTER last_known_addr { host: "127.19.194.62" port: 37647 } }
I20260806 18:38:42.974592 10958 raft_consensus.cc:388] T 00000000000000000000000000000000 P cacda9f469af4273a0e007ddc2c370b1 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 18:38:42.974680 10958 raft_consensus.cc:482] T 00000000000000000000000000000000 P cacda9f469af4273a0e007ddc2c370b1 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 18:38:42.974783 10958 raft_consensus.cc:3037] T 00000000000000000000000000000000 P cacda9f469af4273a0e007ddc2c370b1 [term 0 FOLLOWER]: Advancing to term 1
I20260806 18:38:42.975665 10961 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:38:42.971788 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50825 (local address 127.19.194.62:37647)
0806 18:38:42.971923 (+ 135us) server_negotiation.cc:206] Beginning negotiation
0806 18:38:42.971927 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 18:38:42.972058 (+ 131us) server_negotiation.cc:407] Connection header received
0806 18:38:42.972207 (+ 149us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:38:42.972210 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:38:42.972250 (+ 40us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:38:42.972337 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:38:42.972595 (+ 258us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:38:42.973118 (+ 523us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:38:42.973928 (+ 810us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:38:42.974263 (+ 335us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:38:42.974375 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:38:42.975250 (+ 875us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:38:42.975254 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:38:42.975256 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 18:38:42.975268 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:38:42.975299 (+ 31us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 18:38:42.975309 (+ 10us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:38:42.975388 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0806 18:38:42.975476 (+ 88us) server_negotiation.cc:299] Negotiation successful
0806 18:38:42.975521 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260806 18:38:42.975840 10958 raft_consensus.cc:504] T 00000000000000000000000000000000 P cacda9f469af4273a0e007ddc2c370b1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cacda9f469af4273a0e007ddc2c370b1" member_type: VOTER last_known_addr { host: "127.19.194.62" port: 37647 } }
I20260806 18:38:42.976195 10958 leader_election.cc:302] T 00000000000000000000000000000000 P cacda9f469af4273a0e007ddc2c370b1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: cacda9f469af4273a0e007ddc2c370b1; no voters:
I20260806 18:38:42.976539 10958 leader_election.cc:288] T 00000000000000000000000000000000 P cacda9f469af4273a0e007ddc2c370b1 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 18:38:42.976611 10968 raft_consensus.cc:2781] T 00000000000000000000000000000000 P cacda9f469af4273a0e007ddc2c370b1 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 18:38:42.976893 10968 raft_consensus.cc:686] T 00000000000000000000000000000000 P cacda9f469af4273a0e007ddc2c370b1 [term 1 LEADER]: Becoming Leader. State: Replica: cacda9f469af4273a0e007ddc2c370b1, State: Running, Role: LEADER
I20260806 18:38:42.977228 10968 consensus_queue.cc:237] T 00000000000000000000000000000000 P cacda9f469af4273a0e007ddc2c370b1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cacda9f469af4273a0e007ddc2c370b1" member_type: VOTER last_known_addr { host: "127.19.194.62" port: 37647 } }
I20260806 18:38:42.977754 10958 sys_catalog.cc:564] T 00000000000000000000000000000000 P cacda9f469af4273a0e007ddc2c370b1 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 18:38:42.978986 10969 sys_catalog.cc:455] T 00000000000000000000000000000000 P cacda9f469af4273a0e007ddc2c370b1 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "cacda9f469af4273a0e007ddc2c370b1" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cacda9f469af4273a0e007ddc2c370b1" member_type: VOTER last_known_addr { host: "127.19.194.62" port: 37647 } } }
I20260806 18:38:42.979269 10970 sys_catalog.cc:455] T 00000000000000000000000000000000 P cacda9f469af4273a0e007ddc2c370b1 [sys.catalog]: SysCatalogTable state changed. Reason: New leader cacda9f469af4273a0e007ddc2c370b1. Latest consensus state: current_term: 1 leader_uuid: "cacda9f469af4273a0e007ddc2c370b1" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cacda9f469af4273a0e007ddc2c370b1" member_type: VOTER last_known_addr { host: "127.19.194.62" port: 37647 } } }
I20260806 18:38:42.979369 10970 sys_catalog.cc:458] T 00000000000000000000000000000000 P cacda9f469af4273a0e007ddc2c370b1 [sys.catalog]: This master's current role is: LEADER
I20260806 18:38:42.979581 10969 sys_catalog.cc:458] T 00000000000000000000000000000000 P cacda9f469af4273a0e007ddc2c370b1 [sys.catalog]: This master's current role is: LEADER
I20260806 18:38:42.979879 10983 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 18:38:42.982499 10983 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 18:38:42.987972 10983 catalog_manager.cc:1347] Generated new cluster ID: 1cfec858d2d54f3e9a05acaa5c870139
I20260806 18:38:42.988039 10983 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 18:38:43.004966 10983 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 18:38:43.006273 10983 catalog_manager.cc:1504] Loading token signing keys...
I20260806 18:38:43.013761 10983 catalog_manager.cc:5868] T 00000000000000000000000000000000 P cacda9f469af4273a0e007ddc2c370b1: Generated new TSK 0
I20260806 18:38:43.014555 10983 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 18:38:43.027937 20232 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJW69tn/build/debug/bin/kudu with pid 10886
2026-08-06T18:38:43Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/16 (1435 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/17
Loading random data
Initializing database '/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786041246094603-20232-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 18:38:43 dist-test-slave-1xrh krb5kdc[10994](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 18:38:43 dist-test-slave-1xrh krb5kdc[10994](info): set up 2 sockets
Aug 06 18:38:43 dist-test-slave-1xrh krb5kdc[10994](info): commencing operation
krb5kdc: starting...
W20260806 18:38:44.088753 20232 mini_kdc.cc:121] Time spent starting KDC: real 1.039s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786041246094603-20232-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786041246094603-20232-0/krb5kdc/test-user.keytab.
Aug 06 18:38:44 dist-test-slave-1xrh krb5kdc[10994](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041524, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T18:38:44Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T18:38:44Z Disabled control of system clock
WARNING: no policy specified for kudu/127.19.194.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.19.194.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.19.194.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786041246094603-20232-0/krb5kdc/kudu_127.19.194.62.keytab.
Entry for principal kudu/127.19.194.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786041246094603-20232-0/krb5kdc/kudu_127.19.194.62.keytab.
I20260806 18:38:44.211867 20232 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJW69tn/build/debug/bin/kudu
/tmp/dist-test-taskJW69tn/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786041246094603-20232-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786041246094603-20232-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786041246094603-20232-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786041246094603-20232-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.19.194.62:45801
--webserver_interface=127.19.194.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786041246094603-20232-0/krb5kdc/kudu_127.19.194.62.keytab
--principal=kudu/127.19.194.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.19.194.20:42841
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.19.194.62:45801
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786041246094603-20232-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786041246094603-20232-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786041246094603-20232-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 18:38:44.318723 11007 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 18:38:44.318988 11007 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 18:38:44.319028 11007 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 18:38:44.322911 11007 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 18:38:44.322971 11007 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 18:38:44.322989 11007 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 18:38:44.323004 11007 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 18:38:44.323019 11007 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 18:38:44.328204 11007 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.194.20:42841
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786041246094603-20232-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786041246094603-20232-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.19.194.62:45801
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786041246094603-20232-0/krb5kdc/kudu_127.19.194.62.keytab
--principal=kudu/127.19.194.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.19.194.62:45801
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786041246094603-20232-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.19.194.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11007
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786041246094603-20232-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 17:59:13 UTC on ccycloud.quasar-wvsnzx.root.comops.site
build id 4571028
I20260806 18:38:44.329119 11007 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 18:38:44.330160 11007 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 18:38:44.338886 11015 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:38:44.338893 11012 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:38:44.338893 11013 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 18:38:44.339051 11007 server_base.cc:1034] running on GCE node
I20260806 18:38:44.339710 11007 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 18:38:44.340741 11007 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 18:38:44.341926 11007 hybrid_clock.cc:648] HybridClock initialized: now 1786041524341920 us; error 43 us; skew 500 ppm
Aug 06 18:38:44 dist-test-slave-1xrh krb5kdc[10994](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041524, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.194.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 18:38:44.344682 11007 init.cc:377] Logged in from keytab as kudu/127.19.194.62@KRBTEST.COM (short username kudu)
I20260806 18:38:44.345541 11007 webserver.cc:460] Webserver started at http://127.19.194.62:40563/ using document root <none> and password file <none>
I20260806 18:38:44.346071 11007 fs_manager.cc:359] Metadata directory not provided
I20260806 18:38:44.346159 11007 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 18:38:44.346465 11007 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 18:38:44.348511 11007 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786041246094603-20232-0/minicluster-data/master-0/data/instance:
uuid: "f2888bce704546e4ab4eed024ac28138"
format_stamp: "Formatted at 2026-08-06 18:38:44 on dist-test-slave-1xrh"
server_key: "4cc61828e0a75bc3b94eeeb02fa60e7d"
server_key_iv: "b15834bc25e0318286849c0ed6343ad8"
server_key_version: "encryptionkey@0"
I20260806 18:38:44.349056 11007 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786041246094603-20232-0/minicluster-data/master-0/wal/instance:
uuid: "f2888bce704546e4ab4eed024ac28138"
format_stamp: "Formatted at 2026-08-06 18:38:44 on dist-test-slave-1xrh"
server_key: "4cc61828e0a75bc3b94eeeb02fa60e7d"
server_key_iv: "b15834bc25e0318286849c0ed6343ad8"
server_key_version: "encryptionkey@0"
I20260806 18:38:44.352705 11007 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.005s sys 0.000s
I20260806 18:38:44.355010 11022 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 18:38:44.356217 11007 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260806 18:38:44.356369 11007 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786041246094603-20232-0/minicluster-data/master-0/data,/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786041246094603-20232-0/minicluster-data/master-0/wal
uuid: "f2888bce704546e4ab4eed024ac28138"
format_stamp: "Formatted at 2026-08-06 18:38:44 on dist-test-slave-1xrh"
server_key: "4cc61828e0a75bc3b94eeeb02fa60e7d"
server_key_iv: "b15834bc25e0318286849c0ed6343ad8"
server_key_version: "encryptionkey@0"
I20260806 18:38:44.356487 11007 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786041246094603-20232-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786041246094603-20232-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786041246094603-20232-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 18:38:44.361039 11007 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 18:38:44.363732 11007 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 18:38:44.364020 11007 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 18:38:44.371272 11007 rpc_server.cc:307] RPC server started. Bound to: 127.19.194.62:45801
I20260806 18:38:44.371315 11079 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.194.62:45801 every 8 connection(s)
I20260806 18:38:44.372160 11007 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786041246094603-20232-0/minicluster-data/master-0/data/info.pb
I20260806 18:38:44.374709 11080 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 18:38:44.377578 20232 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJW69tn/build/debug/bin/kudu as pid 11007
I20260806 18:38:44.377720 20232 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786041246094603-20232-0/minicluster-data/master-0/wal/instance
I20260806 18:38:44.378057 20232 external_mini_cluster.cc:1428] Setting key 66ec3202ca8d71e99364c49a058c2457
I20260806 18:38:44.380698 11080 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f2888bce704546e4ab4eed024ac28138: Bootstrap starting.
I20260806 18:38:44.383049 11080 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P f2888bce704546e4ab4eed024ac28138: Neither blocks nor log segments found. Creating new log.
I20260806 18:38:44.383206 11083 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:38:44.379514 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49497 (local address 127.19.194.62:45801)
0806 18:38:44.379935 (+ 421us) server_negotiation.cc:206] Beginning negotiation
0806 18:38:44.379950 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0806 18:38:44.379984 (+ 34us) server_negotiation.cc:407] Connection header received
0806 18:38:44.380573 (+ 589us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:38:44.380588 (+ 15us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:38:44.380908 (+ 320us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:38:44.381226 (+ 318us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:38:44.381988 (+ 762us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:38:44.381993 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:38:44.382004 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 18:38:44.382014 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:38:44.382061 (+ 47us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 18:38:44.382079 (+ 18us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:38:44.382221 (+ 142us) server_negotiation.cc:1035] Waiting for connection context
0806 18:38:44.382290 (+ 69us) server_negotiation.cc:299] Negotiation successful
0806 18:38:44.382440 (+ 150us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":290,"thread_start_us":132,"threads_started":1}
I20260806 18:38:44.384019 11080 log.cc:824] T 00000000000000000000000000000000 P f2888bce704546e4ab4eed024ac28138: Log is configured to *not* fsync() on all Append() calls
I20260806 18:38:44.386266 20232 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260806 18:38:44.386276 11080 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f2888bce704546e4ab4eed024ac28138: No bootstrap required, opened a new log
I20260806 18:38:44.389627 11083 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:38:44.388230 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59153 (local address 127.19.194.62:45801)
0806 18:38:44.388332 (+ 102us) server_negotiation.cc:206] Beginning negotiation
0806 18:38:44.388336 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 18:38:44.388479 (+ 143us) server_negotiation.cc:407] Connection header received
0806 18:38:44.388557 (+ 78us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:38:44.388559 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:38:44.388589 (+ 30us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:38:44.388669 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:38:44.389280 (+ 611us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:38:44.389284 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:38:44.389286 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 18:38:44.389297 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:38:44.389324 (+ 27us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 18:38:44.389335 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:38:44.389420 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0806 18:38:44.389475 (+ 55us) server_negotiation.cc:299] Negotiation successful
0806 18:38:44.389506 (+ 31us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260806 18:38:44.390170 11080 raft_consensus.cc:348] T 00000000000000000000000000000000 P f2888bce704546e4ab4eed024ac28138 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f2888bce704546e4ab4eed024ac28138" member_type: VOTER last_known_addr { host: "127.19.194.62" port: 45801 } }
I20260806 18:38:44.390430 11080 raft_consensus.cc:374] T 00000000000000000000000000000000 P f2888bce704546e4ab4eed024ac28138 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 18:38:44.390486 11080 raft_consensus.cc:729] T 00000000000000000000000000000000 P f2888bce704546e4ab4eed024ac28138 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f2888bce704546e4ab4eed024ac28138, State: Initialized, Role: FOLLOWER
I20260806 18:38:44.390846 11080 consensus_queue.cc:260] T 00000000000000000000000000000000 P f2888bce704546e4ab4eed024ac28138 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f2888bce704546e4ab4eed024ac28138" member_type: VOTER last_known_addr { host: "127.19.194.62" port: 45801 } }
I20260806 18:38:44.390991 11080 raft_consensus.cc:388] T 00000000000000000000000000000000 P f2888bce704546e4ab4eed024ac28138 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 18:38:44.391073 11080 raft_consensus.cc:482] T 00000000000000000000000000000000 P f2888bce704546e4ab4eed024ac28138 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 18:38:44.391180 11080 raft_consensus.cc:3037] T 00000000000000000000000000000000 P f2888bce704546e4ab4eed024ac28138 [term 0 FOLLOWER]: Advancing to term 1
I20260806 18:38:44.392020 11080 raft_consensus.cc:504] T 00000000000000000000000000000000 P f2888bce704546e4ab4eed024ac28138 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f2888bce704546e4ab4eed024ac28138" member_type: VOTER last_known_addr { host: "127.19.194.62" port: 45801 } }
I20260806 18:38:44.392333 11080 leader_election.cc:302] T 00000000000000000000000000000000 P f2888bce704546e4ab4eed024ac28138 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: f2888bce704546e4ab4eed024ac28138; no voters:
I20260806 18:38:44.392613 11080 leader_election.cc:288] T 00000000000000000000000000000000 P f2888bce704546e4ab4eed024ac28138 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 18:38:44.392742 11090 raft_consensus.cc:2781] T 00000000000000000000000000000000 P f2888bce704546e4ab4eed024ac28138 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 18:38:44.393002 11090 raft_consensus.cc:686] T 00000000000000000000000000000000 P f2888bce704546e4ab4eed024ac28138 [term 1 LEADER]: Becoming Leader. State: Replica: f2888bce704546e4ab4eed024ac28138, State: Running, Role: LEADER
I20260806 18:38:44.393314 11090 consensus_queue.cc:237] T 00000000000000000000000000000000 P f2888bce704546e4ab4eed024ac28138 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f2888bce704546e4ab4eed024ac28138" member_type: VOTER last_known_addr { host: "127.19.194.62" port: 45801 } }
I20260806 18:38:44.394063 11080 sys_catalog.cc:564] T 00000000000000000000000000000000 P f2888bce704546e4ab4eed024ac28138 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 18:38:44.394481 11090 sys_catalog.cc:455] T 00000000000000000000000000000000 P f2888bce704546e4ab4eed024ac28138 [sys.catalog]: SysCatalogTable state changed. Reason: New leader f2888bce704546e4ab4eed024ac28138. Latest consensus state: current_term: 1 leader_uuid: "f2888bce704546e4ab4eed024ac28138" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f2888bce704546e4ab4eed024ac28138" member_type: VOTER last_known_addr { host: "127.19.194.62" port: 45801 } } }
I20260806 18:38:44.394610 11090 sys_catalog.cc:458] T 00000000000000000000000000000000 P f2888bce704546e4ab4eed024ac28138 [sys.catalog]: This master's current role is: LEADER
I20260806 18:38:44.394925 11092 sys_catalog.cc:455] T 00000000000000000000000000000000 P f2888bce704546e4ab4eed024ac28138 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "f2888bce704546e4ab4eed024ac28138" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f2888bce704546e4ab4eed024ac28138" member_type: VOTER last_known_addr { host: "127.19.194.62" port: 45801 } } }
I20260806 18:38:44.395063 11092 sys_catalog.cc:458] T 00000000000000000000000000000000 P f2888bce704546e4ab4eed024ac28138 [sys.catalog]: This master's current role is: LEADER
I20260806 18:38:44.395002 11095 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 18:38:44.398032 11095 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 18:38:44.402959 11095 catalog_manager.cc:1347] Generated new cluster ID: 3883f4814f994020a76afceffc68a4cf
I20260806 18:38:44.403030 11095 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 18:38:44.410722 11095 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 18:38:44.411006 11095 catalog_manager.cc:1504] Loading token signing keys...
I20260806 18:38:44.426435 11095 catalog_manager.cc:5868] T 00000000000000000000000000000000 P f2888bce704546e4ab4eed024ac28138: Generated new TSK 0
I20260806 18:38:44.427178 11095 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 18:38:44.441177 20232 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJW69tn/build/debug/bin/kudu with pid 11007
2026-08-06T18:38:44Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/17 (1412 ms)
[----------] 18 tests from AuthTokenIssuingTest (25656 ms total)
[----------] 2 tests from ConnectToFollowerMasterTest
[ RUN ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/0
Loading random data
Initializing database '/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786041246094603-20232-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 18:38:44 dist-test-slave-1xrh krb5kdc[11113](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 18:38:44 dist-test-slave-1xrh krb5kdc[11113](info): set up 2 sockets
Aug 06 18:38:44 dist-test-slave-1xrh krb5kdc[11113](info): commencing operation
krb5kdc: starting...
W20260806 18:38:45.488416 20232 mini_kdc.cc:121] Time spent starting KDC: real 1.026s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786041246094603-20232-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786041246094603-20232-0/krb5kdc/test-user.keytab.
Aug 06 18:38:45 dist-test-slave-1xrh krb5kdc[11113](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041525, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T18:38:45Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T18:38:45Z Disabled control of system clock
WARNING: no policy specified for kudu/127.19.194.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.19.194.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.19.194.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786041246094603-20232-0/krb5kdc/kudu_127.19.194.62.keytab.
Entry for principal kudu/127.19.194.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786041246094603-20232-0/krb5kdc/kudu_127.19.194.62.keytab.
I20260806 18:38:45.609993 20232 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJW69tn/build/debug/bin/kudu
/tmp/dist-test-taskJW69tn/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786041246094603-20232-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786041246094603-20232-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786041246094603-20232-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786041246094603-20232-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.19.194.62:39303
--webserver_interface=127.19.194.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786041246094603-20232-0/krb5kdc/kudu_127.19.194.62.keytab
--principal=kudu/127.19.194.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.19.194.20:43589
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.19.194.62:39303,127.19.194.61:44923,127.19.194.60:43081
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786041246094603-20232-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786041246094603-20232-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786041246094603-20232-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 18:38:45.717708 11128 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260806 18:38:45.717957 11128 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 18:38:45.717978 11128 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 18:38:45.718012 11128 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 18:38:45.721791 11128 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 18:38:45.721840 11128 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 18:38:45.721858 11128 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 18:38:45.721874 11128 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 18:38:45.721889 11128 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 18:38:45.727022 11128 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.194.20:43589
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786041246094603-20232-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786041246094603-20232-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.19.194.62:39303,127.19.194.61:44923,127.19.194.60:43081
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786041246094603-20232-0/krb5kdc/kudu_127.19.194.62.keytab
--principal=kudu/127.19.194.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.19.194.62:39303
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786041246094603-20232-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.19.194.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11128
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786041246094603-20232-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 17:59:13 UTC on ccycloud.quasar-wvsnzx.root.comops.site
build id 4571028
I20260806 18:38:45.728010 11128 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 18:38:45.728905 11128 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 18:38:45.737141 11133 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:38:45.737138 11134 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 18:38:45.737274 11128 server_base.cc:1034] running on GCE node
W20260806 18:38:45.737190 11136 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 18:38:45.737967 11128 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 18:38:45.738976 11128 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 18:38:45.740135 11128 hybrid_clock.cc:648] HybridClock initialized: now 1786041525740115 us; error 43 us; skew 500 ppm
Aug 06 18:38:45 dist-test-slave-1xrh krb5kdc[11113](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041525, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.194.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 18:38:45.742769 11128 init.cc:377] Logged in from keytab as kudu/127.19.194.62@KRBTEST.COM (short username kudu)
I20260806 18:38:45.743649 11128 webserver.cc:460] Webserver started at http://127.19.194.62:45597/ using document root <none> and password file <none>
I20260806 18:38:45.744184 11128 fs_manager.cc:359] Metadata directory not provided
I20260806 18:38:45.744243 11128 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 18:38:45.744485 11128 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 18:38:45.746414 11128 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786041246094603-20232-0/minicluster-data/master-0/data/instance:
uuid: "633c13ed73414fa2a52bb4ce636a9e95"
format_stamp: "Formatted at 2026-08-06 18:38:45 on dist-test-slave-1xrh"
server_key: "27d757e2a220744255fe4ae687779f00"
server_key_iv: "180b69c6433b4e47e05199793ca2d4a0"
server_key_version: "encryptionkey@0"
I20260806 18:38:45.746976 11128 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786041246094603-20232-0/minicluster-data/master-0/wal/instance:
uuid: "633c13ed73414fa2a52bb4ce636a9e95"
format_stamp: "Formatted at 2026-08-06 18:38:45 on dist-test-slave-1xrh"
server_key: "27d757e2a220744255fe4ae687779f00"
server_key_iv: "180b69c6433b4e47e05199793ca2d4a0"
server_key_version: "encryptionkey@0"
I20260806 18:38:45.750488 11128 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.002s sys 0.000s
I20260806 18:38:45.752723 11145 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 18:38:45.753892 11128 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.000s sys 0.001s
I20260806 18:38:45.754038 11128 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786041246094603-20232-0/minicluster-data/master-0/data,/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786041246094603-20232-0/minicluster-data/master-0/wal
uuid: "633c13ed73414fa2a52bb4ce636a9e95"
format_stamp: "Formatted at 2026-08-06 18:38:45 on dist-test-slave-1xrh"
server_key: "27d757e2a220744255fe4ae687779f00"
server_key_iv: "180b69c6433b4e47e05199793ca2d4a0"
server_key_version: "encryptionkey@0"
I20260806 18:38:45.754146 11128 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786041246094603-20232-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786041246094603-20232-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786041246094603-20232-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 18:38:45.778410 11128 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 18:38:45.781157 11128 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 18:38:45.781371 11128 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 18:38:45.788610 11128 rpc_server.cc:307] RPC server started. Bound to: 127.19.194.62:39303
I20260806 18:38:45.788890 11200 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.194.62:39303 every 8 connection(s)
I20260806 18:38:45.789578 11128 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786041246094603-20232-0/minicluster-data/master-0/data/info.pb
I20260806 18:38:45.792263 11201 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 18:38:45.795625 20232 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJW69tn/build/debug/bin/kudu as pid 11128
I20260806 18:38:45.795749 20232 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786041246094603-20232-0/minicluster-data/master-0/wal/instance
I20260806 18:38:45.796025 20232 external_mini_cluster.cc:1428] Setting key 0dfd7dc8880a5e687fd460ccad5db52a
I20260806 18:38:45.795993 11201 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.19.194.62" port: 39303 } has no permanent_uuid. Determining permanent_uuid...
WARNING: no policy specified for kudu/127.19.194.61@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.19.194.61@KRBTEST.COM" created.
Aug 06 18:38:45 dist-test-slave-1xrh krb5kdc[11113](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041525, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.194.62@KRBTEST.COM for kudu/127.19.194.62@KRBTEST.COM
I20260806 18:38:45.814209 11205 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:38:45.798400 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.194.62:39303 (local address 127.0.0.1:40544)
0806 18:38:45.798987 (+ 587us) negotiation.cc:107] Waiting for socket to connect
0806 18:38:45.799033 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0806 18:38:45.799834 (+ 801us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:38:45.800639 (+ 805us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:38:45.800644 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:38:45.800989 (+ 345us) client_negotiation.cc:189] Negotiated authn=SASL
0806 18:38:45.801288 (+ 299us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:38:45.801301 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:38:45.803061 (+ 1760us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:38:45.803063 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:38:45.803838 (+ 775us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:38:45.803847 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:38:45.804727 (+ 880us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:38:45.804730 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:38:45.804818 (+ 88us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:38:45.805441 (+ 623us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 18:38:45.805456 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0806 18:38:45.809168 (+ 3712us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 18:38:45.811438 (+ 2270us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:38:45.811448 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:38:45.811456 (+ 8us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:38:45.811661 (+ 205us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:38:45.811965 (+ 304us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:38:45.811969 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:38:45.811971 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:38:45.812070 (+ 99us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:38:45.812642 (+ 572us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 18:38:45.812648 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 18:38:45.812790 (+ 142us) client_negotiation.cc:769] Sending connection context
0806 18:38:45.812944 (+ 154us) client_negotiation.cc:240] Negotiation successful
0806 18:38:45.813290 (+ 346us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":387,"mutex_wait_us":142,"spinlock_wait_cycles":29056,"thread_start_us":165,"threads_started":1}
I20260806 18:38:45.814209 11204 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:38:45.798400 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40544 (local address 127.19.194.62:39303)
0806 18:38:45.798848 (+ 448us) server_negotiation.cc:206] Beginning negotiation
0806 18:38:45.798868 (+ 20us) server_negotiation.cc:399] Waiting for connection header
0806 18:38:45.799173 (+ 305us) server_negotiation.cc:407] Connection header received
0806 18:38:45.800131 (+ 958us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:38:45.800139 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:38:45.800390 (+ 251us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:38:45.800593 (+ 203us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:38:45.801436 (+ 843us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:38:45.802939 (+ 1503us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:38:45.803962 (+ 1023us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:38:45.804579 (+ 617us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:38:45.804770 (+ 191us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:38:45.809312 (+ 4542us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:38:45.809317 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:38:45.809326 (+ 9us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 18:38:45.809345 (+ 19us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:38:45.811338 (+ 1993us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:38:45.811767 (+ 429us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:38:45.811772 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:38:45.811774 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:38:45.811870 (+ 96us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:38:45.812162 (+ 292us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:38:45.812164 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:38:45.812166 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:38:45.812530 (+ 364us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:38:45.812676 (+ 146us) server_negotiation.cc:1035] Waiting for connection context
0806 18:38:45.813017 (+ 341us) server_negotiation.cc:299] Negotiation successful
0806 18:38:45.813238 (+ 221us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":51,"server-negotiator.queue_time_us":287,"thread_start_us":118,"threads_started":1}
I20260806 18:38:45.816744 11201 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.19.194.61" port: 44923 } has no permanent_uuid. Determining permanent_uuid...
I20260806 18:38:45.817526 11205 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:38:45.817193 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.194.61:44923 (local address 127.0.0.1:48740)
0806 18:38:45.817306 (+ 113us) negotiation.cc:107] Waiting for socket to connect
0806 18:38:45.817447 (+ 141us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.194.61:44923: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":46}
W20260806 18:38:45.817696 11147 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.19.194.61:44923: connect: Connection refused (error 111)
I20260806 18:38:45.818528 11205 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:38:45.818311 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.194.61:44923 (local address 127.0.0.1:48746)
0806 18:38:45.818438 (+ 127us) negotiation.cc:107] Waiting for socket to connect
0806 18:38:45.818481 (+ 43us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.194.61:44923: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":54}
W20260806 18:38:45.818698 11201 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.19.194.61:44923: Network error: Client connection negotiation failed: client connection to 127.19.194.61:44923: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.19.194.61 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786041246094603-20232-0/krb5kdc/kudu_127.19.194.61.keytab.
Entry for principal kudu/127.19.194.61 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786041246094603-20232-0/krb5kdc/kudu_127.19.194.61.keytab.
I20260806 18:38:45.825630 20232 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJW69tn/build/debug/bin/kudu
/tmp/dist-test-taskJW69tn/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786041246094603-20232-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786041246094603-20232-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786041246094603-20232-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786041246094603-20232-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.19.194.61:44923
--webserver_interface=127.19.194.61
--webserver_port=0
--keytab_file=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786041246094603-20232-0/krb5kdc/kudu_127.19.194.61.keytab
--principal=kudu/127.19.194.61
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.19.194.20:43589
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.19.194.62:39303,127.19.194.61:44923,127.19.194.60:43081
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786041246094603-20232-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786041246094603-20232-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786041246094603-20232-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260806 18:38:45.867959 11201 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.19.194.61" port: 44923 } attempt: 1
I20260806 18:38:45.868911 11205 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:38:45.868616 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.194.61:44923 (local address 127.0.0.1:48762)
0806 18:38:45.868768 (+ 152us) negotiation.cc:107] Waiting for socket to connect
0806 18:38:45.868830 (+ 62us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.194.61:44923: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":60}
I20260806 18:38:45.869725 11205 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:38:45.869435 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.194.61:44923 (local address 127.0.0.1:48766)
0806 18:38:45.869591 (+ 156us) negotiation.cc:107] Waiting for socket to connect
0806 18:38:45.869658 (+ 67us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.194.61:44923: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":84}
W20260806 18:38:45.869890 11201 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.19.194.61:44923: Network error: Client connection negotiation failed: client connection to 127.19.194.61:44923: connect: Connection refused (error 111)
W20260806 18:38:45.933183 11208 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260806 18:38:45.933450 11208 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 18:38:45.933475 11208 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 18:38:45.933560 11208 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 18:38:45.937477 11208 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 18:38:45.937532 11208 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 18:38:45.937551 11208 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 18:38:45.937565 11208 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 18:38:45.937580 11208 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 18:38:45.938050 11201 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.19.194.61" port: 44923 } attempt: 2
I20260806 18:38:45.938908 11205 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:38:45.938564 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.194.61:44923 (local address 127.0.0.1:48774)
0806 18:38:45.938745 (+ 181us) negotiation.cc:107] Waiting for socket to connect
0806 18:38:45.938817 (+ 72us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.194.61:44923: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":72}
I20260806 18:38:45.939636 11205 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:38:45.939425 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.194.61:44923 (local address 127.0.0.1:48782)
0806 18:38:45.939527 (+ 102us) negotiation.cc:107] Waiting for socket to connect
0806 18:38:45.939574 (+ 47us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.194.61:44923: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":41}
W20260806 18:38:45.939767 11201 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.19.194.61:44923: Network error: Client connection negotiation failed: client connection to 127.19.194.61:44923: connect: Connection refused (error 111)
I20260806 18:38:45.943271 11208 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.194.20:43589
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786041246094603-20232-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786041246094603-20232-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.19.194.62:39303,127.19.194.61:44923,127.19.194.60:43081
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786041246094603-20232-0/krb5kdc/kudu_127.19.194.61.keytab
--principal=kudu/127.19.194.61
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.19.194.61:44923
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786041246094603-20232-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.19.194.61
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11208
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786041246094603-20232-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 17:59:13 UTC on ccycloud.quasar-wvsnzx.root.comops.site
build id 4571028
I20260806 18:38:45.944183 11208 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 18:38:45.945078 11208 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 18:38:45.953249 11214 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:38:45.953308 11215 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:38:45.953259 11217 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 18:38:45.953399 11208 server_base.cc:1034] running on GCE node
I20260806 18:38:45.954061 11208 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 18:38:45.955075 11208 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 18:38:45.956286 11208 hybrid_clock.cc:648] HybridClock initialized: now 1786041525956265 us; error 30 us; skew 500 ppm
Aug 06 18:38:45 dist-test-slave-1xrh krb5kdc[11113](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041525, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.194.61@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 18:38:45.959105 11208 init.cc:377] Logged in from keytab as kudu/127.19.194.61@KRBTEST.COM (short username kudu)
I20260806 18:38:45.959978 11208 webserver.cc:460] Webserver started at http://127.19.194.61:37437/ using document root <none> and password file <none>
I20260806 18:38:45.960642 11208 fs_manager.cc:359] Metadata directory not provided
I20260806 18:38:45.960726 11208 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 18:38:45.960994 11208 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 18:38:45.963865 11208 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786041246094603-20232-0/minicluster-data/master-1/data/instance:
uuid: "67fb66439d324a3293b9b659938172b3"
format_stamp: "Formatted at 2026-08-06 18:38:45 on dist-test-slave-1xrh"
server_key: "726f020fe15aee35a1964696036cf9a5"
server_key_iv: "1af574d11761a147321c91f30d3cce43"
server_key_version: "encryptionkey@0"
I20260806 18:38:45.964650 11208 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786041246094603-20232-0/minicluster-data/master-1/wal/instance:
uuid: "67fb66439d324a3293b9b659938172b3"
format_stamp: "Formatted at 2026-08-06 18:38:45 on dist-test-slave-1xrh"
server_key: "726f020fe15aee35a1964696036cf9a5"
server_key_iv: "1af574d11761a147321c91f30d3cce43"
server_key_version: "encryptionkey@0"
I20260806 18:38:45.969626 11208 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260806 18:38:45.972219 11225 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 18:38:45.973505 11208 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260806 18:38:45.973642 11208 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786041246094603-20232-0/minicluster-data/master-1/data,/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786041246094603-20232-0/minicluster-data/master-1/wal
uuid: "67fb66439d324a3293b9b659938172b3"
format_stamp: "Formatted at 2026-08-06 18:38:45 on dist-test-slave-1xrh"
server_key: "726f020fe15aee35a1964696036cf9a5"
server_key_iv: "1af574d11761a147321c91f30d3cce43"
server_key_version: "encryptionkey@0"
I20260806 18:38:45.973752 11208 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786041246094603-20232-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786041246094603-20232-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786041246094603-20232-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 18:38:45.987262 11208 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 18:38:45.989849 11208 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 18:38:45.990048 11208 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 18:38:45.996956 11208 rpc_server.cc:307] RPC server started. Bound to: 127.19.194.61:44923
I20260806 18:38:45.997254 11282 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.194.61:44923 every 8 connection(s)
I20260806 18:38:45.997922 11208 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786041246094603-20232-0/minicluster-data/master-1/data/info.pb
I20260806 18:38:46.000538 11283 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 18:38:46.001473 20232 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJW69tn/build/debug/bin/kudu as pid 11208
I20260806 18:38:46.001595 20232 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786041246094603-20232-0/minicluster-data/master-1/wal/instance
I20260806 18:38:46.001919 20232 external_mini_cluster.cc:1428] Setting key 58452825cb70c41f8bbc6cbc2946d38f
I20260806 18:38:46.005290 11283 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.19.194.62" port: 39303 } has no permanent_uuid. Determining permanent_uuid...
WARNING: no policy specified for kudu/127.19.194.60@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.19.194.60@KRBTEST.COM" created.
Aug 06 18:38:46 dist-test-slave-1xrh krb5kdc[11113](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041525, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.194.61@KRBTEST.COM for kudu/127.19.194.62@KRBTEST.COM
I20260806 18:38:46.020145 11204 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:38:46.007116 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40554 (local address 127.19.194.62:39303)
0806 18:38:46.007291 (+ 175us) server_negotiation.cc:206] Beginning negotiation
0806 18:38:46.007295 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 18:38:46.008102 (+ 807us) server_negotiation.cc:407] Connection header received
0806 18:38:46.009028 (+ 926us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:38:46.009034 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:38:46.009106 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:38:46.009216 (+ 110us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:38:46.010330 (+ 1114us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:38:46.011231 (+ 901us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:38:46.012512 (+ 1281us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:38:46.012743 (+ 231us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:38:46.012826 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:38:46.015770 (+ 2944us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:38:46.015773 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:38:46.015775 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 18:38:46.015799 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:38:46.017858 (+ 2059us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:38:46.018488 (+ 630us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:38:46.018491 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:38:46.018492 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:38:46.018562 (+ 70us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:38:46.018957 (+ 395us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:38:46.018959 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:38:46.018960 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:38:46.019159 (+ 199us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:38:46.019250 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0806 18:38:46.019919 (+ 669us) server_negotiation.cc:299] Negotiation successful
0806 18:38:46.020023 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260806 18:38:46.021266 11286 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:38:46.007492 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.194.62:39303 (local address 127.0.0.1:40554)
0806 18:38:46.007932 (+ 440us) negotiation.cc:107] Waiting for socket to connect
0806 18:38:46.007983 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0806 18:38:46.008821 (+ 838us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:38:46.009359 (+ 538us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:38:46.009365 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:38:46.009689 (+ 324us) client_negotiation.cc:189] Negotiated authn=SASL
0806 18:38:46.010168 (+ 479us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:38:46.010181 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:38:46.011349 (+ 1168us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:38:46.011352 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:38:46.012397 (+ 1045us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:38:46.012408 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:38:46.012920 (+ 512us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:38:46.012926 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:38:46.013074 (+ 148us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:38:46.013728 (+ 654us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 18:38:46.013744 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0806 18:38:46.015596 (+ 1852us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 18:38:46.017988 (+ 2392us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:38:46.017997 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:38:46.018013 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:38:46.018391 (+ 378us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:38:46.018657 (+ 266us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:38:46.018662 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:38:46.018666 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:38:46.018856 (+ 190us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:38:46.019280 (+ 424us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 18:38:46.019289 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 18:38:46.019616 (+ 327us) client_negotiation.cc:769] Sending connection context
0806 18:38:46.019867 (+ 251us) client_negotiation.cc:240] Negotiation successful
0806 18:38:46.020213 (+ 346us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":245,"thread_start_us":133,"threads_started":1}
I20260806 18:38:46.022626 11283 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.19.194.61" port: 44923 } has no permanent_uuid. Determining permanent_uuid...
Aug 06 18:38:46 dist-test-slave-1xrh krb5kdc[11113](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041525, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.194.61@KRBTEST.COM for kudu/127.19.194.61@KRBTEST.COM
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.19.194.60 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786041246094603-20232-0/krb5kdc/kudu_127.19.194.60.keytab.
Entry for principal kudu/127.19.194.60 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786041246094603-20232-0/krb5kdc/kudu_127.19.194.60.keytab.
I20260806 18:38:46.030804 20232 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJW69tn/build/debug/bin/kudu
/tmp/dist-test-taskJW69tn/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786041246094603-20232-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786041246094603-20232-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786041246094603-20232-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786041246094603-20232-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.19.194.60:43081
--webserver_interface=127.19.194.60
--webserver_port=0
--keytab_file=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786041246094603-20232-0/krb5kdc/kudu_127.19.194.60.keytab
--principal=kudu/127.19.194.60
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.19.194.20:43589
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.19.194.62:39303,127.19.194.61:44923,127.19.194.60:43081
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786041246094603-20232-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786041246094603-20232-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786041246094603-20232-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260806 18:38:46.030936 11201 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.19.194.61" port: 44923 } attempt: 3
I20260806 18:38:46.032294 11286 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:38:46.023111 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.194.61:44923 (local address 127.0.0.1:48788)
0806 18:38:46.023257 (+ 146us) negotiation.cc:107] Waiting for socket to connect
0806 18:38:46.023272 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0806 18:38:46.023344 (+ 72us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:38:46.023938 (+ 594us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:38:46.023941 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:38:46.024069 (+ 128us) client_negotiation.cc:189] Negotiated authn=SASL
0806 18:38:46.024151 (+ 82us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:38:46.024156 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:38:46.025117 (+ 961us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:38:46.025120 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:38:46.025643 (+ 523us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:38:46.025651 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:38:46.026275 (+ 624us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:38:46.026277 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:38:46.026384 (+ 107us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:38:46.026828 (+ 444us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 18:38:46.026840 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0806 18:38:46.028189 (+ 1349us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 18:38:46.030296 (+ 2107us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:38:46.030300 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:38:46.030301 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:38:46.030566 (+ 265us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:38:46.030892 (+ 326us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:38:46.030897 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:38:46.030901 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:38:46.031002 (+ 101us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:38:46.031648 (+ 646us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 18:38:46.031653 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 18:38:46.031756 (+ 103us) client_negotiation.cc:769] Sending connection context
0806 18:38:46.031871 (+ 115us) client_negotiation.cc:240] Negotiation successful
0806 18:38:46.032026 (+ 155us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":82}
I20260806 18:38:46.032407 11288 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:38:46.023235 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48788 (local address 127.19.194.61:44923)
0806 18:38:46.023523 (+ 288us) server_negotiation.cc:206] Beginning negotiation
0806 18:38:46.023532 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0806 18:38:46.023548 (+ 16us) server_negotiation.cc:407] Connection header received
0806 18:38:46.023577 (+ 29us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:38:46.023580 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:38:46.023756 (+ 176us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:38:46.023937 (+ 181us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:38:46.024233 (+ 296us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:38:46.025010 (+ 777us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:38:46.025744 (+ 734us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:38:46.026172 (+ 428us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:38:46.026255 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:38:46.028312 (+ 2057us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:38:46.028321 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:38:46.028331 (+ 10us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 18:38:46.028352 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:38:46.030168 (+ 1816us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:38:46.030683 (+ 515us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:38:46.030688 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:38:46.030689 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:38:46.030767 (+ 78us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:38:46.031112 (+ 345us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:38:46.031140 (+ 28us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:38:46.031145 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:38:46.031475 (+ 330us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:38:46.031608 (+ 133us) server_negotiation.cc:1035] Waiting for connection context
0806 18:38:46.031992 (+ 384us) server_negotiation.cc:299] Negotiation successful
0806 18:38:46.032143 (+ 151us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":212,"thread_start_us":99,"threads_started":1}
I20260806 18:38:46.034163 11283 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.19.194.60" port: 43081 } has no permanent_uuid. Determining permanent_uuid...
I20260806 18:38:46.034952 11286 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:38:46.034635 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.194.60:43081 (local address 127.0.0.1:43362)
0806 18:38:46.034769 (+ 134us) negotiation.cc:107] Waiting for socket to connect
0806 18:38:46.034862 (+ 93us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.194.60:43081: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":53}
W20260806 18:38:46.035624 11229 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.19.194.60:43081: connect: Connection refused (error 111)
I20260806 18:38:46.036438 11286 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:38:46.036226 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.194.60:43081 (local address 127.0.0.1:43378)
0806 18:38:46.036328 (+ 102us) negotiation.cc:107] Waiting for socket to connect
0806 18:38:46.036378 (+ 50us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.194.60:43081: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":43}
W20260806 18:38:46.036648 11283 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.19.194.60:43081: Network error: Client connection negotiation failed: client connection to 127.19.194.60:43081: connect: Connection refused (error 111)
Aug 06 18:38:46 dist-test-slave-1xrh krb5kdc[11113](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041525, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.194.62@KRBTEST.COM for kudu/127.19.194.61@KRBTEST.COM
I20260806 18:38:46.041152 11289 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:38:46.031497 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48800 (local address 127.19.194.61:44923)
0806 18:38:46.031970 (+ 473us) server_negotiation.cc:206] Beginning negotiation
0806 18:38:46.031978 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 18:38:46.031990 (+ 12us) server_negotiation.cc:407] Connection header received
0806 18:38:46.032018 (+ 28us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:38:46.032021 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:38:46.032054 (+ 33us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:38:46.032899 (+ 845us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:38:46.032975 (+ 76us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:38:46.033756 (+ 781us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:38:46.034779 (+ 1023us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:38:46.035046 (+ 267us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:38:46.035191 (+ 145us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:38:46.037888 (+ 2697us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:38:46.037891 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:38:46.037892 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 18:38:46.037912 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:38:46.039466 (+ 1554us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:38:46.039990 (+ 524us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:38:46.039993 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:38:46.039994 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:38:46.040062 (+ 68us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:38:46.040375 (+ 313us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:38:46.040377 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:38:46.040378 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:38:46.040529 (+ 151us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:38:46.040600 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0806 18:38:46.040961 (+ 361us) server_negotiation.cc:299] Negotiation successful
0806 18:38:46.041047 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":399,"thread_start_us":144,"threads_started":1}
I20260806 18:38:46.041302 11205 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:38:46.031459 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.194.61:44923 (local address 127.0.0.1:48800)
0806 18:38:46.031654 (+ 195us) negotiation.cc:107] Waiting for socket to connect
0806 18:38:46.031671 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0806 18:38:46.031757 (+ 86us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:38:46.032201 (+ 444us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:38:46.032205 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:38:46.032386 (+ 181us) client_negotiation.cc:189] Negotiated authn=SASL
0806 18:38:46.032500 (+ 114us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:38:46.032508 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:38:46.033893 (+ 1385us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:38:46.033897 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:38:46.034640 (+ 743us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:38:46.034653 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:38:46.035214 (+ 561us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:38:46.035220 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:38:46.035347 (+ 127us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:38:46.036090 (+ 743us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 18:38:46.036109 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0806 18:38:46.037719 (+ 1610us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 18:38:46.039579 (+ 1860us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:38:46.039585 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:38:46.039589 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:38:46.039889 (+ 300us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:38:46.040158 (+ 269us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:38:46.040163 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:38:46.040167 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:38:46.040282 (+ 115us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:38:46.040630 (+ 348us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 18:38:46.040636 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 18:38:46.040763 (+ 127us) client_negotiation.cc:769] Sending connection context
0806 18:38:46.040906 (+ 143us) client_negotiation.cc:240] Negotiation successful
0806 18:38:46.041070 (+ 164us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":111}
I20260806 18:38:46.042065 11201 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.19.194.60" port: 43081 } has no permanent_uuid. Determining permanent_uuid...
I20260806 18:38:46.042743 11205 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:38:46.042529 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.194.60:43081 (local address 127.0.0.1:43390)
0806 18:38:46.042641 (+ 112us) negotiation.cc:107] Waiting for socket to connect
0806 18:38:46.042686 (+ 45us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.194.60:43081: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":49}
I20260806 18:38:46.043514 11205 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:38:46.043271 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.194.60:43081 (local address 127.0.0.1:43402)
0806 18:38:46.043421 (+ 150us) negotiation.cc:107] Waiting for socket to connect
0806 18:38:46.043464 (+ 43us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.194.60:43081: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":92}
W20260806 18:38:46.043716 11201 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.19.194.60:43081: Network error: Client connection negotiation failed: client connection to 127.19.194.60:43081: connect: Connection refused (error 111)
I20260806 18:38:46.074903 11201 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.19.194.60" port: 43081 } attempt: 1
I20260806 18:38:46.075711 11205 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:38:46.075451 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.194.60:43081 (local address 127.0.0.1:43410)
0806 18:38:46.075585 (+ 134us) negotiation.cc:107] Waiting for socket to connect
0806 18:38:46.075640 (+ 55us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.194.60:43081: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":53}
I20260806 18:38:46.076448 11205 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:38:46.076263 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.194.60:43081 (local address 127.0.0.1:43420)
0806 18:38:46.076360 (+ 97us) negotiation.cc:107] Waiting for socket to connect
0806 18:38:46.076400 (+ 40us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.194.60:43081: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":43}
W20260806 18:38:46.076625 11201 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.19.194.60:43081: Network error: Client connection negotiation failed: client connection to 127.19.194.60:43081: connect: Connection refused (error 111)
I20260806 18:38:46.085886 11283 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.19.194.60" port: 43081 } attempt: 1
I20260806 18:38:46.086544 11286 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:38:46.086313 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.194.60:43081 (local address 127.0.0.1:43428)
0806 18:38:46.086438 (+ 125us) negotiation.cc:107] Waiting for socket to connect
0806 18:38:46.086483 (+ 45us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.194.60:43081: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":50}
I20260806 18:38:46.087236 11286 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:38:46.086988 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.194.60:43081 (local address 127.0.0.1:43438)
0806 18:38:46.087085 (+ 97us) negotiation.cc:107] Waiting for socket to connect
0806 18:38:46.087179 (+ 94us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.194.60:43081: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":38}
W20260806 18:38:46.087356 11283 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.19.194.60:43081: Network error: Client connection negotiation failed: client connection to 127.19.194.60:43081: connect: Connection refused (error 111)
W20260806 18:38:46.145541 11291 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260806 18:38:46.145776 11291 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 18:38:46.145800 11291 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 18:38:46.145835 11291 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 18:38:46.149698 11291 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 18:38:46.149744 11291 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 18:38:46.149775 11291 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 18:38:46.149791 11291 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 18:38:46.149806 11291 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 18:38:46.151773 11201 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.19.194.60" port: 43081 } attempt: 2
I20260806 18:38:46.152563 11205 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:38:46.152288 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.194.60:43081 (local address 127.0.0.1:43446)
0806 18:38:46.152434 (+ 146us) negotiation.cc:107] Waiting for socket to connect
0806 18:38:46.152492 (+ 58us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.194.60:43081: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":57}
I20260806 18:38:46.153298 11205 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:38:46.153112 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.194.60:43081 (local address 127.0.0.1:43448)
0806 18:38:46.153207 (+ 95us) negotiation.cc:107] Waiting for socket to connect
0806 18:38:46.153248 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.194.60:43081: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":41}
W20260806 18:38:46.153484 11201 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.19.194.60:43081: Network error: Client connection negotiation failed: client connection to 127.19.194.60:43081: connect: Connection refused (error 111)
I20260806 18:38:46.155293 11291 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.194.20:43589
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786041246094603-20232-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786041246094603-20232-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.19.194.62:39303,127.19.194.61:44923,127.19.194.60:43081
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786041246094603-20232-0/krb5kdc/kudu_127.19.194.60.keytab
--principal=kudu/127.19.194.60
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.19.194.60:43081
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786041246094603-20232-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.19.194.60
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11291
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786041246094603-20232-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 17:59:13 UTC on ccycloud.quasar-wvsnzx.root.comops.site
build id 4571028
I20260806 18:38:46.155515 11283 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.19.194.60" port: 43081 } attempt: 2
I20260806 18:38:46.156210 11286 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:38:46.155977 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.194.60:43081 (local address 127.0.0.1:43464)
0806 18:38:46.156101 (+ 124us) negotiation.cc:107] Waiting for socket to connect
0806 18:38:46.156148 (+ 47us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.194.60:43081: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":52}
I20260806 18:38:46.156459 11291 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 18:38:46.156854 11286 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:38:46.156661 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.194.60:43081 (local address 127.0.0.1:43468)
0806 18:38:46.156760 (+ 99us) negotiation.cc:107] Waiting for socket to connect
0806 18:38:46.156807 (+ 47us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.194.60:43081: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":39}
W20260806 18:38:46.156975 11283 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.19.194.60:43081: Network error: Client connection negotiation failed: client connection to 127.19.194.60:43081: connect: Connection refused (error 111)
I20260806 18:38:46.157559 11291 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 18:38:46.165584 11298 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:38:46.165624 11297 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:38:46.165622 11301 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 18:38:46.165773 11291 server_base.cc:1034] running on GCE node
I20260806 18:38:46.166409 11291 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 18:38:46.167477 11291 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 18:38:46.168654 11291 hybrid_clock.cc:648] HybridClock initialized: now 1786041526168624 us; error 38 us; skew 500 ppm
Aug 06 18:38:46 dist-test-slave-1xrh krb5kdc[11113](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041526, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.194.60@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 18:38:46.171317 11291 init.cc:377] Logged in from keytab as kudu/127.19.194.60@KRBTEST.COM (short username kudu)
I20260806 18:38:46.172180 11291 webserver.cc:460] Webserver started at http://127.19.194.60:33557/ using document root <none> and password file <none>
I20260806 18:38:46.172701 11291 fs_manager.cc:359] Metadata directory not provided
I20260806 18:38:46.172804 11291 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 18:38:46.173027 11291 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 18:38:46.174913 11291 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786041246094603-20232-0/minicluster-data/master-2/data/instance:
uuid: "9bc603ee6c8946b9a7ab114f807854d3"
format_stamp: "Formatted at 2026-08-06 18:38:46 on dist-test-slave-1xrh"
server_key: "8fc962ea20e30b9349a4028d8561cd7a"
server_key_iv: "2da80c05856034e2586428ac0fffe5cb"
server_key_version: "encryptionkey@0"
I20260806 18:38:46.175557 11291 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786041246094603-20232-0/minicluster-data/master-2/wal/instance:
uuid: "9bc603ee6c8946b9a7ab114f807854d3"
format_stamp: "Formatted at 2026-08-06 18:38:46 on dist-test-slave-1xrh"
server_key: "8fc962ea20e30b9349a4028d8561cd7a"
server_key_iv: "2da80c05856034e2586428ac0fffe5cb"
server_key_version: "encryptionkey@0"
I20260806 18:38:46.179023 11291 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.000s
I20260806 18:38:46.181167 11308 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 18:38:46.182283 11291 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260806 18:38:46.182423 11291 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786041246094603-20232-0/minicluster-data/master-2/data,/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786041246094603-20232-0/minicluster-data/master-2/wal
uuid: "9bc603ee6c8946b9a7ab114f807854d3"
format_stamp: "Formatted at 2026-08-06 18:38:46 on dist-test-slave-1xrh"
server_key: "8fc962ea20e30b9349a4028d8561cd7a"
server_key_iv: "2da80c05856034e2586428ac0fffe5cb"
server_key_version: "encryptionkey@0"
I20260806 18:38:46.182529 11291 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786041246094603-20232-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786041246094603-20232-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786041246094603-20232-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 18:38:46.197680 11291 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 18:38:46.200629 11291 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 18:38:46.200850 11291 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 18:38:46.207710 11291 rpc_server.cc:307] RPC server started. Bound to: 127.19.194.60:43081
I20260806 18:38:46.208016 11362 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.194.60:43081 every 8 connection(s)
I20260806 18:38:46.208639 11291 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786041246094603-20232-0/minicluster-data/master-2/data/info.pb
I20260806 18:38:46.209343 20232 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJW69tn/build/debug/bin/kudu as pid 11291
I20260806 18:38:46.209478 20232 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786041246094603-20232-0/minicluster-data/master-2/wal/instance
I20260806 18:38:46.209771 20232 external_mini_cluster.cc:1428] Setting key a5e348c00ac921b9638e28a7af4be750
I20260806 18:38:46.211781 11363 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 18:38:46.215878 11363 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.19.194.62" port: 39303 } has no permanent_uuid. Determining permanent_uuid...
Aug 06 18:38:46 dist-test-slave-1xrh krb5kdc[11113](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041525, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.19.194.60@KRBTEST.COM
Aug 06 18:38:46 dist-test-slave-1xrh krb5kdc[11113](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041526, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.194.60@KRBTEST.COM for kudu/127.19.194.62@KRBTEST.COM
I20260806 18:38:46.224155 11366 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:38:46.211051 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43478 (local address 127.19.194.60:43081)
0806 18:38:46.211502 (+ 451us) server_negotiation.cc:206] Beginning negotiation
0806 18:38:46.211517 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0806 18:38:46.211570 (+ 53us) server_negotiation.cc:407] Connection header received
0806 18:38:46.212286 (+ 716us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:38:46.212304 (+ 18us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:38:46.212615 (+ 311us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:38:46.212891 (+ 276us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:38:46.213833 (+ 942us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:38:46.214834 (+ 1001us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:38:46.216051 (+ 1217us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:38:46.216570 (+ 519us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:38:46.216692 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:38:46.219461 (+ 2769us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:38:46.219467 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:38:46.219476 (+ 9us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 18:38:46.219503 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:38:46.221184 (+ 1681us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:38:46.221633 (+ 449us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:38:46.221643 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:38:46.221647 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:38:46.221753 (+ 106us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:38:46.222041 (+ 288us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:38:46.222046 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:38:46.222048 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:38:46.222446 (+ 398us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:38:46.222643 (+ 197us) server_negotiation.cc:1035] Waiting for connection context
0806 18:38:46.222872 (+ 229us) server_negotiation.cc:299] Negotiation successful
0806 18:38:46.223234 (+ 362us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":316,"thread_start_us":146,"threads_started":1}
I20260806 18:38:46.226994 11367 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:38:46.217816 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.194.62:39303 (local address 127.0.0.1:40562)
0806 18:38:46.218270 (+ 454us) negotiation.cc:107] Waiting for socket to connect
0806 18:38:46.218317 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0806 18:38:46.218459 (+ 142us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:38:46.218759 (+ 300us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:38:46.218764 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:38:46.219191 (+ 427us) client_negotiation.cc:189] Negotiated authn=SASL
0806 18:38:46.219333 (+ 142us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:38:46.219339 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:38:46.220231 (+ 892us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:38:46.220234 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:38:46.220823 (+ 589us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:38:46.220832 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:38:46.221292 (+ 460us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:38:46.221296 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:38:46.221423 (+ 127us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:38:46.221907 (+ 484us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 18:38:46.221919 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0806 18:38:46.223430 (+ 1511us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 18:38:46.225231 (+ 1801us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:38:46.225240 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:38:46.225245 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:38:46.225480 (+ 235us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:38:46.225768 (+ 288us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:38:46.225772 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:38:46.225773 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:38:46.225857 (+ 84us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:38:46.226432 (+ 575us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 18:38:46.226438 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 18:38:46.226574 (+ 136us) client_negotiation.cc:769] Sending connection context
0806 18:38:46.226722 (+ 148us) client_negotiation.cc:240] Negotiation successful
0806 18:38:46.226850 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":264,"thread_start_us":171,"threads_started":1}
I20260806 18:38:46.227022 11204 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:38:46.217893 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40562 (local address 127.19.194.62:39303)
0806 18:38:46.218051 (+ 158us) server_negotiation.cc:206] Beginning negotiation
0806 18:38:46.218056 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 18:38:46.218399 (+ 343us) server_negotiation.cc:407] Connection header received
0806 18:38:46.218570 (+ 171us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:38:46.218573 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:38:46.218623 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:38:46.218709 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:38:46.219458 (+ 749us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:38:46.220094 (+ 636us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:38:46.220963 (+ 869us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:38:46.221176 (+ 213us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:38:46.221277 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:38:46.223561 (+ 2284us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:38:46.223564 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:38:46.223566 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 18:38:46.223591 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:38:46.225120 (+ 1529us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:38:46.225585 (+ 465us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:38:46.225589 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:38:46.225591 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:38:46.225679 (+ 88us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:38:46.226027 (+ 348us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:38:46.226030 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:38:46.226031 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:38:46.226309 (+ 278us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:38:46.226412 (+ 103us) server_negotiation.cc:1035] Waiting for connection context
0806 18:38:46.226803 (+ 391us) server_negotiation.cc:299] Negotiation successful
0806 18:38:46.226907 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260806 18:38:46.228018 11363 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.19.194.61" port: 44923 } has no permanent_uuid. Determining permanent_uuid...
Aug 06 18:38:46 dist-test-slave-1xrh krb5kdc[11113](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041525, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.19.194.62@KRBTEST.COM
Aug 06 18:38:46 dist-test-slave-1xrh krb5kdc[11113](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041526, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.194.60@KRBTEST.COM for kudu/127.19.194.61@KRBTEST.COM
I20260806 18:38:46.235899 11368 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:38:46.226403 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40570 (local address 127.19.194.62:39303)
0806 18:38:46.226769 (+ 366us) server_negotiation.cc:206] Beginning negotiation
0806 18:38:46.226777 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 18:38:46.227344 (+ 567us) server_negotiation.cc:407] Connection header received
0806 18:38:46.227588 (+ 244us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:38:46.227590 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:38:46.227625 (+ 35us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:38:46.227724 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:38:46.228324 (+ 600us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:38:46.228934 (+ 610us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:38:46.229701 (+ 767us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:38:46.229941 (+ 240us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:38:46.230060 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:38:46.232318 (+ 2258us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:38:46.232321 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:38:46.232323 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 18:38:46.232344 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:38:46.234085 (+ 1741us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:38:46.234564 (+ 479us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:38:46.234568 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:38:46.234573 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:38:46.234673 (+ 100us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:38:46.234923 (+ 250us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:38:46.234927 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:38:46.234928 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:38:46.235191 (+ 263us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:38:46.235365 (+ 174us) server_negotiation.cc:1035] Waiting for connection context
0806 18:38:46.235516 (+ 151us) server_negotiation.cc:299] Negotiation successful
0806 18:38:46.235628 (+ 112us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":275,"thread_start_us":126,"threads_started":1}
I20260806 18:38:46.237017 11367 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:38:46.228507 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.194.61:44923 (local address 127.0.0.1:48816)
0806 18:38:46.228644 (+ 137us) negotiation.cc:107] Waiting for socket to connect
0806 18:38:46.228671 (+ 27us) client_negotiation.cc:174] Beginning negotiation
0806 18:38:46.228745 (+ 74us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:38:46.229001 (+ 256us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:38:46.229004 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:38:46.229166 (+ 162us) client_negotiation.cc:189] Negotiated authn=SASL
0806 18:38:46.229268 (+ 102us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:38:46.229273 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:38:46.230031 (+ 758us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:38:46.230036 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:38:46.230713 (+ 677us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:38:46.230722 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:38:46.231259 (+ 537us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:38:46.231263 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:38:46.231370 (+ 107us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:38:46.231832 (+ 462us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 18:38:46.231869 (+ 37us) client_negotiation.cc:656] calling sasl_client_start()
0806 18:38:46.232994 (+ 1125us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 18:38:46.235214 (+ 2220us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:38:46.235217 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:38:46.235218 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:38:46.235394 (+ 176us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:38:46.235687 (+ 293us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:38:46.235691 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:38:46.235693 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:38:46.235772 (+ 79us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:38:46.236503 (+ 731us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 18:38:46.236505 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 18:38:46.236613 (+ 108us) client_negotiation.cc:769] Sending connection context
0806 18:38:46.236722 (+ 109us) client_negotiation.cc:240] Negotiation successful
0806 18:38:46.236851 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":89}
I20260806 18:38:46.237236 11289 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:38:46.228622 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48816 (local address 127.19.194.61:44923)
0806 18:38:46.228768 (+ 146us) server_negotiation.cc:206] Beginning negotiation
0806 18:38:46.228774 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 18:38:46.228792 (+ 18us) server_negotiation.cc:407] Connection header received
0806 18:38:46.228836 (+ 44us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:38:46.228838 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:38:46.228887 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:38:46.228987 (+ 100us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:38:46.229388 (+ 401us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:38:46.229905 (+ 517us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:38:46.230833 (+ 928us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:38:46.231096 (+ 263us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:38:46.231262 (+ 166us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:38:46.233371 (+ 2109us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:38:46.233376 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:38:46.233380 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 18:38:46.233416 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:38:46.235065 (+ 1649us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:38:46.235502 (+ 437us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:38:46.235507 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:38:46.235509 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:38:46.235592 (+ 83us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:38:46.235867 (+ 275us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:38:46.235871 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:38:46.235873 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:38:46.236122 (+ 249us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:38:46.236460 (+ 338us) server_negotiation.cc:1035] Waiting for connection context
0806 18:38:46.236798 (+ 338us) server_negotiation.cc:299] Negotiation successful
0806 18:38:46.236936 (+ 138us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260806 18:38:46.237848 11363 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.19.194.60" port: 43081 } has no permanent_uuid. Determining permanent_uuid...
Aug 06 18:38:46 dist-test-slave-1xrh krb5kdc[11113](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041525, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.19.194.61@KRBTEST.COM
Aug 06 18:38:46 dist-test-slave-1xrh krb5kdc[11113](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041526, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.194.60@KRBTEST.COM for kudu/127.19.194.60@KRBTEST.COM
I20260806 18:38:46.246132 11288 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:38:46.237026 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48822 (local address 127.19.194.61:44923)
0806 18:38:46.237158 (+ 132us) server_negotiation.cc:206] Beginning negotiation
0806 18:38:46.237166 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 18:38:46.237182 (+ 16us) server_negotiation.cc:407] Connection header received
0806 18:38:46.237345 (+ 163us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:38:46.237349 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:38:46.237411 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:38:46.237487 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:38:46.238188 (+ 701us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:38:46.238940 (+ 752us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:38:46.239878 (+ 938us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:38:46.240116 (+ 238us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:38:46.240192 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:38:46.242632 (+ 2440us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:38:46.242635 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:38:46.242637 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 18:38:46.242657 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:38:46.244384 (+ 1727us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:38:46.244853 (+ 469us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:38:46.244856 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:38:46.244857 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:38:46.244927 (+ 70us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:38:46.245242 (+ 315us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:38:46.245245 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:38:46.245247 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:38:46.245428 (+ 181us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:38:46.245635 (+ 207us) server_negotiation.cc:1035] Waiting for connection context
0806 18:38:46.245778 (+ 143us) server_negotiation.cc:299] Negotiation successful
0806 18:38:46.245903 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260806 18:38:46.246388 11367 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:38:46.238321 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.194.60:43081 (local address 127.0.0.1:43490)
0806 18:38:46.238414 (+ 93us) negotiation.cc:107] Waiting for socket to connect
0806 18:38:46.238427 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0806 18:38:46.238493 (+ 66us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:38:46.238686 (+ 193us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:38:46.238689 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:38:46.238839 (+ 150us) client_negotiation.cc:189] Negotiated authn=SASL
0806 18:38:46.238920 (+ 81us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:38:46.238934 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:38:46.239690 (+ 756us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:38:46.239693 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:38:46.240334 (+ 641us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:38:46.240346 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:38:46.241394 (+ 1048us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:38:46.241397 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:38:46.241497 (+ 100us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:38:46.241998 (+ 501us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 18:38:46.242010 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0806 18:38:46.243211 (+ 1201us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 18:38:46.244993 (+ 1782us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:38:46.244995 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:38:46.244997 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:38:46.245175 (+ 178us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:38:46.245480 (+ 305us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:38:46.245482 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:38:46.245484 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:38:46.245564 (+ 80us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:38:46.245909 (+ 345us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 18:38:46.245911 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 18:38:46.245988 (+ 77us) client_negotiation.cc:769] Sending connection context
0806 18:38:46.246099 (+ 111us) client_negotiation.cc:240] Negotiation successful
0806 18:38:46.246223 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":29}
I20260806 18:38:46.246623 11366 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:38:46.238366 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43490 (local address 127.19.194.60:43081)
0806 18:38:46.238470 (+ 104us) server_negotiation.cc:206] Beginning negotiation
0806 18:38:46.238474 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 18:38:46.238494 (+ 20us) server_negotiation.cc:407] Connection header received
0806 18:38:46.238568 (+ 74us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:38:46.238570 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:38:46.238600 (+ 30us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:38:46.238661 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:38:46.239042 (+ 381us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:38:46.239585 (+ 543us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:38:46.240466 (+ 881us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:38:46.241245 (+ 779us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:38:46.241375 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:38:46.243345 (+ 1970us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:38:46.243348 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:38:46.243349 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 18:38:46.243368 (+ 19us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:38:46.244866 (+ 1498us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:38:46.245266 (+ 400us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:38:46.245271 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:38:46.245273 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:38:46.245343 (+ 70us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:38:46.245658 (+ 315us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:38:46.245661 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:38:46.245663 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:38:46.245815 (+ 152us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:38:46.245924 (+ 109us) server_negotiation.cc:1035] Waiting for connection context
0806 18:38:46.246154 (+ 230us) server_negotiation.cc:299] Negotiation successful
0806 18:38:46.246362 (+ 208us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260806 18:38:46.248129 11283 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.19.194.60" port: 43081 } attempt: 3
I20260806 18:38:46.249743 20232 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260806 18:38:46.250133 11363 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9bc603ee6c8946b9a7ab114f807854d3: Bootstrap starting.
I20260806 18:38:46.251175 11174 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "00000000000000000000000000000000"
dest_uuid: "633c13ed73414fa2a52bb4ce636a9e95"
from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:40570
I20260806 18:38:46.252650 11201 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.19.194.60" port: 43081 } attempt: 3
Aug 06 18:38:46 dist-test-slave-1xrh krb5kdc[11113](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041525, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.194.61@KRBTEST.COM for kudu/127.19.194.60@KRBTEST.COM
I20260806 18:38:46.253526 11363 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 9bc603ee6c8946b9a7ab114f807854d3: Neither blocks nor log segments found. Creating new log.
I20260806 18:38:46.254448 11363 log.cc:824] T 00000000000000000000000000000000 P 9bc603ee6c8946b9a7ab114f807854d3: Log is configured to *not* fsync() on all Append() calls
I20260806 18:38:46.256632 11363 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9bc603ee6c8946b9a7ab114f807854d3: No bootstrap required, opened a new log
I20260806 18:38:46.257149 11286 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:38:46.248580 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.194.60:43081 (local address 127.0.0.1:43502)
0806 18:38:46.248724 (+ 144us) negotiation.cc:107] Waiting for socket to connect
0806 18:38:46.248737 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0806 18:38:46.248806 (+ 69us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:38:46.249197 (+ 391us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:38:46.249200 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:38:46.249390 (+ 190us) client_negotiation.cc:189] Negotiated authn=SASL
0806 18:38:46.249474 (+ 84us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:38:46.249481 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:38:46.250431 (+ 950us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:38:46.250435 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:38:46.251203 (+ 768us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:38:46.251213 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:38:46.251690 (+ 477us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:38:46.251693 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:38:46.251772 (+ 79us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:38:46.252202 (+ 430us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 18:38:46.252215 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0806 18:38:46.253671 (+ 1456us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 18:38:46.255608 (+ 1937us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:38:46.255611 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:38:46.255613 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:38:46.255843 (+ 230us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:38:46.256143 (+ 300us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:38:46.256146 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:38:46.256148 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:38:46.256234 (+ 86us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:38:46.256676 (+ 442us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 18:38:46.256680 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 18:38:46.256769 (+ 89us) client_negotiation.cc:769] Sending connection context
0806 18:38:46.256877 (+ 108us) client_negotiation.cc:240] Negotiation successful
0806 18:38:46.257015 (+ 138us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":85}
I20260806 18:38:46.257216 11366 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:38:46.248649 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43502 (local address 127.19.194.60:43081)
0806 18:38:46.248750 (+ 101us) server_negotiation.cc:206] Beginning negotiation
0806 18:38:46.248756 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 18:38:46.248770 (+ 14us) server_negotiation.cc:407] Connection header received
0806 18:38:46.249053 (+ 283us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:38:46.249056 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:38:46.249105 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:38:46.249178 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:38:46.249602 (+ 424us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:38:46.250277 (+ 675us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:38:46.251356 (+ 1079us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:38:46.251571 (+ 215us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:38:46.251685 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:38:46.253813 (+ 2128us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:38:46.253816 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:38:46.253818 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 18:38:46.253837 (+ 19us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:38:46.255495 (+ 1658us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:38:46.255951 (+ 456us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:38:46.255954 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:38:46.255956 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:38:46.256037 (+ 81us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:38:46.256323 (+ 286us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:38:46.256327 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:38:46.256331 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:38:46.256562 (+ 231us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:38:46.256657 (+ 95us) server_negotiation.cc:1035] Waiting for connection context
0806 18:38:46.256951 (+ 294us) server_negotiation.cc:299] Negotiation successful
0806 18:38:46.257077 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260806 18:38:46.260015 11363 raft_consensus.cc:348] T 00000000000000000000000000000000 P 9bc603ee6c8946b9a7ab114f807854d3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "633c13ed73414fa2a52bb4ce636a9e95" member_type: VOTER last_known_addr { host: "127.19.194.62" port: 39303 } } peers { permanent_uuid: "67fb66439d324a3293b9b659938172b3" member_type: VOTER last_known_addr { host: "127.19.194.61" port: 44923 } } peers { permanent_uuid: "9bc603ee6c8946b9a7ab114f807854d3" member_type: VOTER last_known_addr { host: "127.19.194.60" port: 43081 } }
I20260806 18:38:46.260129 11283 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 67fb66439d324a3293b9b659938172b3: Bootstrap starting.
I20260806 18:38:46.260185 11363 raft_consensus.cc:729] T 00000000000000000000000000000000 P 9bc603ee6c8946b9a7ab114f807854d3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9bc603ee6c8946b9a7ab114f807854d3, State: Initialized, Role: FOLLOWER
I20260806 18:38:46.260421 11363 consensus_queue.cc:260] T 00000000000000000000000000000000 P 9bc603ee6c8946b9a7ab114f807854d3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "633c13ed73414fa2a52bb4ce636a9e95" member_type: VOTER last_known_addr { host: "127.19.194.62" port: 39303 } } peers { permanent_uuid: "67fb66439d324a3293b9b659938172b3" member_type: VOTER last_known_addr { host: "127.19.194.61" port: 44923 } } peers { permanent_uuid: "9bc603ee6c8946b9a7ab114f807854d3" member_type: VOTER last_known_addr { host: "127.19.194.60" port: 43081 } }
I20260806 18:38:46.260924 11372 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9bc603ee6c8946b9a7ab114f807854d3 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "633c13ed73414fa2a52bb4ce636a9e95" member_type: VOTER last_known_addr { host: "127.19.194.62" port: 39303 } } peers { permanent_uuid: "67fb66439d324a3293b9b659938172b3" member_type: VOTER last_known_addr { host: "127.19.194.61" port: 44923 } } peers { permanent_uuid: "9bc603ee6c8946b9a7ab114f807854d3" member_type: VOTER last_known_addr { host: "127.19.194.60" port: 43081 } } }
I20260806 18:38:46.261216 11372 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9bc603ee6c8946b9a7ab114f807854d3 [sys.catalog]: This master's current role is: FOLLOWER
I20260806 18:38:46.261415 11363 sys_catalog.cc:564] T 00000000000000000000000000000000 P 9bc603ee6c8946b9a7ab114f807854d3 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 18:38:46.262702 11283 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 67fb66439d324a3293b9b659938172b3: Neither blocks nor log segments found. Creating new log.
I20260806 18:38:46.263799 11283 log.cc:824] T 00000000000000000000000000000000 P 67fb66439d324a3293b9b659938172b3: Log is configured to *not* fsync() on all Append() calls
Aug 06 18:38:46 dist-test-slave-1xrh krb5kdc[11113](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041525, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.194.62@KRBTEST.COM for kudu/127.19.194.60@KRBTEST.COM
W20260806 18:38:46.267261 11384 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 9bc603ee6c8946b9a7ab114f807854d3: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260806 18:38:46.267395 11384 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260806 18:38:46.264919 11291 master_runner.cc:186] Error getting master registration for 127.19.194.62:39303: OK, instance_id { permanent_uuid: "633c13ed73414fa2a52bb4ce636a9e95" instance_seqno: 1786041525786489 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260806 18:38:46.268841 11291 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260806 18:38:46.269466 11283 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 67fb66439d324a3293b9b659938172b3: No bootstrap required, opened a new log
I20260806 18:38:46.271025 11205 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:38:46.253054 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.194.60:43081 (local address 127.0.0.1:43508)
0806 18:38:46.253227 (+ 173us) negotiation.cc:107] Waiting for socket to connect
0806 18:38:46.253240 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0806 18:38:46.253315 (+ 75us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:38:46.262116 (+ 8801us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:38:46.262120 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:38:46.262335 (+ 215us) client_negotiation.cc:189] Negotiated authn=SASL
0806 18:38:46.262459 (+ 124us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:38:46.262469 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:38:46.263704 (+ 1235us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:38:46.263711 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:38:46.264434 (+ 723us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:38:46.264442 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:38:46.265063 (+ 621us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:38:46.265067 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:38:46.265153 (+ 86us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:38:46.265624 (+ 471us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 18:38:46.265638 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0806 18:38:46.267071 (+ 1433us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 18:38:46.269302 (+ 2231us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:38:46.269305 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:38:46.269325 (+ 20us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:38:46.269598 (+ 273us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:38:46.269959 (+ 361us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:38:46.269962 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:38:46.269964 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:38:46.270053 (+ 89us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:38:46.270527 (+ 474us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 18:38:46.270529 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 18:38:46.270621 (+ 92us) client_negotiation.cc:769] Sending connection context
0806 18:38:46.270720 (+ 99us) client_negotiation.cc:240] Negotiation successful
0806 18:38:46.270859 (+ 139us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":106}
I20260806 18:38:46.271543 11370 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:38:46.253131 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43508 (local address 127.19.194.60:43081)
0806 18:38:46.255057 (+ 1926us) server_negotiation.cc:206] Beginning negotiation
0806 18:38:46.255067 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0806 18:38:46.255081 (+ 14us) server_negotiation.cc:407] Connection header received
0806 18:38:46.255116 (+ 35us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:38:46.255144 (+ 28us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:38:46.255196 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:38:46.262589 (+ 7393us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:38:46.262701 (+ 112us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:38:46.263556 (+ 855us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:38:46.264554 (+ 998us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:38:46.264914 (+ 360us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:38:46.265020 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:38:46.267299 (+ 2279us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:38:46.267303 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:38:46.267306 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 18:38:46.267331 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:38:46.269145 (+ 1814us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:38:46.269716 (+ 571us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:38:46.269719 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:38:46.269721 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:38:46.269847 (+ 126us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:38:46.270158 (+ 311us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:38:46.270162 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:38:46.270163 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:38:46.270375 (+ 212us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:38:46.270470 (+ 95us) server_negotiation.cc:1035] Waiting for connection context
0806 18:38:46.271247 (+ 777us) server_negotiation.cc:299] Negotiation successful
0806 18:38:46.271365 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1858,"thread_start_us":91,"threads_started":1}
I20260806 18:38:46.273350 11283 raft_consensus.cc:348] T 00000000000000000000000000000000 P 67fb66439d324a3293b9b659938172b3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "633c13ed73414fa2a52bb4ce636a9e95" member_type: VOTER last_known_addr { host: "127.19.194.62" port: 39303 } } peers { permanent_uuid: "67fb66439d324a3293b9b659938172b3" member_type: VOTER last_known_addr { host: "127.19.194.61" port: 44923 } } peers { permanent_uuid: "9bc603ee6c8946b9a7ab114f807854d3" member_type: VOTER last_known_addr { host: "127.19.194.60" port: 43081 } }
I20260806 18:38:46.273553 11283 raft_consensus.cc:729] T 00000000000000000000000000000000 P 67fb66439d324a3293b9b659938172b3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 67fb66439d324a3293b9b659938172b3, State: Initialized, Role: FOLLOWER
I20260806 18:38:46.273864 11283 consensus_queue.cc:260] T 00000000000000000000000000000000 P 67fb66439d324a3293b9b659938172b3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "633c13ed73414fa2a52bb4ce636a9e95" member_type: VOTER last_known_addr { host: "127.19.194.62" port: 39303 } } peers { permanent_uuid: "67fb66439d324a3293b9b659938172b3" member_type: VOTER last_known_addr { host: "127.19.194.61" port: 44923 } } peers { permanent_uuid: "9bc603ee6c8946b9a7ab114f807854d3" member_type: VOTER last_known_addr { host: "127.19.194.60" port: 43081 } }
I20260806 18:38:46.274322 11388 sys_catalog.cc:455] T 00000000000000000000000000000000 P 67fb66439d324a3293b9b659938172b3 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "633c13ed73414fa2a52bb4ce636a9e95" member_type: VOTER last_known_addr { host: "127.19.194.62" port: 39303 } } peers { permanent_uuid: "67fb66439d324a3293b9b659938172b3" member_type: VOTER last_known_addr { host: "127.19.194.61" port: 44923 } } peers { permanent_uuid: "9bc603ee6c8946b9a7ab114f807854d3" member_type: VOTER last_known_addr { host: "127.19.194.60" port: 43081 } } }
I20260806 18:38:46.274426 11201 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 633c13ed73414fa2a52bb4ce636a9e95: Bootstrap starting.
I20260806 18:38:46.274544 11388 sys_catalog.cc:458] T 00000000000000000000000000000000 P 67fb66439d324a3293b9b659938172b3 [sys.catalog]: This master's current role is: FOLLOWER
I20260806 18:38:46.275079 11283 sys_catalog.cc:564] T 00000000000000000000000000000000 P 67fb66439d324a3293b9b659938172b3 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 18:38:46.277165 11201 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 633c13ed73414fa2a52bb4ce636a9e95: Neither blocks nor log segments found. Creating new log.
I20260806 18:38:46.278010 11201 log.cc:824] T 00000000000000000000000000000000 P 633c13ed73414fa2a52bb4ce636a9e95: Log is configured to *not* fsync() on all Append() calls
I20260806 18:38:46.280323 11201 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 633c13ed73414fa2a52bb4ce636a9e95: No bootstrap required, opened a new log
W20260806 18:38:46.280725 11400 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 67fb66439d324a3293b9b659938172b3: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260806 18:38:46.280788 11400 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260806 18:38:46.279282 11208 master_runner.cc:186] Error getting master registration for 127.19.194.62:39303: OK, instance_id { permanent_uuid: "633c13ed73414fa2a52bb4ce636a9e95" instance_seqno: 1786041525786489 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260806 18:38:46.282495 11208 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260806 18:38:46.283315 11201 raft_consensus.cc:348] T 00000000000000000000000000000000 P 633c13ed73414fa2a52bb4ce636a9e95 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "633c13ed73414fa2a52bb4ce636a9e95" member_type: VOTER last_known_addr { host: "127.19.194.62" port: 39303 } } peers { permanent_uuid: "67fb66439d324a3293b9b659938172b3" member_type: VOTER last_known_addr { host: "127.19.194.61" port: 44923 } } peers { permanent_uuid: "9bc603ee6c8946b9a7ab114f807854d3" member_type: VOTER last_known_addr { host: "127.19.194.60" port: 43081 } }
I20260806 18:38:46.283520 11201 raft_consensus.cc:729] T 00000000000000000000000000000000 P 633c13ed73414fa2a52bb4ce636a9e95 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 633c13ed73414fa2a52bb4ce636a9e95, State: Initialized, Role: FOLLOWER
I20260806 18:38:46.283839 11201 consensus_queue.cc:260] T 00000000000000000000000000000000 P 633c13ed73414fa2a52bb4ce636a9e95 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "633c13ed73414fa2a52bb4ce636a9e95" member_type: VOTER last_known_addr { host: "127.19.194.62" port: 39303 } } peers { permanent_uuid: "67fb66439d324a3293b9b659938172b3" member_type: VOTER last_known_addr { host: "127.19.194.61" port: 44923 } } peers { permanent_uuid: "9bc603ee6c8946b9a7ab114f807854d3" member_type: VOTER last_known_addr { host: "127.19.194.60" port: 43081 } }
I20260806 18:38:46.284420 11403 sys_catalog.cc:455] T 00000000000000000000000000000000 P 633c13ed73414fa2a52bb4ce636a9e95 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "633c13ed73414fa2a52bb4ce636a9e95" member_type: VOTER last_known_addr { host: "127.19.194.62" port: 39303 } } peers { permanent_uuid: "67fb66439d324a3293b9b659938172b3" member_type: VOTER last_known_addr { host: "127.19.194.61" port: 44923 } } peers { permanent_uuid: "9bc603ee6c8946b9a7ab114f807854d3" member_type: VOTER last_known_addr { host: "127.19.194.60" port: 43081 } } }
I20260806 18:38:46.284740 11403 sys_catalog.cc:458] T 00000000000000000000000000000000 P 633c13ed73414fa2a52bb4ce636a9e95 [sys.catalog]: This master's current role is: FOLLOWER
I20260806 18:38:46.285130 11201 sys_catalog.cc:564] T 00000000000000000000000000000000 P 633c13ed73414fa2a52bb4ce636a9e95 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 18:38:46.285207 11174 raft_consensus.cc:482] T 00000000000000000000000000000000 P 633c13ed73414fa2a52bb4ce636a9e95 [term 0 FOLLOWER]: Starting forced leader election (received explicit request)
I20260806 18:38:46.285317 11174 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 633c13ed73414fa2a52bb4ce636a9e95 [term 0 FOLLOWER]: Advancing to term 1
I20260806 18:38:46.286615 11174 raft_consensus.cc:504] T 00000000000000000000000000000000 P 633c13ed73414fa2a52bb4ce636a9e95 [term 1 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "633c13ed73414fa2a52bb4ce636a9e95" member_type: VOTER last_known_addr { host: "127.19.194.62" port: 39303 } } peers { permanent_uuid: "67fb66439d324a3293b9b659938172b3" member_type: VOTER last_known_addr { host: "127.19.194.61" port: 44923 } } peers { permanent_uuid: "9bc603ee6c8946b9a7ab114f807854d3" member_type: VOTER last_known_addr { host: "127.19.194.60" port: 43081 } }
I20260806 18:38:46.288177 11256 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "633c13ed73414fa2a52bb4ce636a9e95" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "67fb66439d324a3293b9b659938172b3"
I20260806 18:38:46.288183 11335 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "633c13ed73414fa2a52bb4ce636a9e95" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "9bc603ee6c8946b9a7ab114f807854d3"
I20260806 18:38:46.288414 11256 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 67fb66439d324a3293b9b659938172b3 [term 0 FOLLOWER]: Advancing to term 1
I20260806 18:38:46.288437 11335 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 9bc603ee6c8946b9a7ab114f807854d3 [term 0 FOLLOWER]: Advancing to term 1
I20260806 18:38:46.289670 11256 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 67fb66439d324a3293b9b659938172b3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 633c13ed73414fa2a52bb4ce636a9e95 in term 1.
I20260806 18:38:46.289796 11335 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 9bc603ee6c8946b9a7ab114f807854d3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 633c13ed73414fa2a52bb4ce636a9e95 in term 1.
I20260806 18:38:46.287602 11174 leader_election.cc:288] T 00000000000000000000000000000000 P 633c13ed73414fa2a52bb4ce636a9e95 [CANDIDATE]: Term 1 election: Requested vote from peers 67fb66439d324a3293b9b659938172b3 (127.19.194.61:44923), 9bc603ee6c8946b9a7ab114f807854d3 (127.19.194.60:43081)
I20260806 18:38:46.290311 11149 leader_election.cc:302] T 00000000000000000000000000000000 P 633c13ed73414fa2a52bb4ce636a9e95 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 633c13ed73414fa2a52bb4ce636a9e95, 9bc603ee6c8946b9a7ab114f807854d3; no voters:
W20260806 18:38:46.291409 11414 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 633c13ed73414fa2a52bb4ce636a9e95: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260806 18:38:46.291478 11414 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260806 18:38:46.292371 11403 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 633c13ed73414fa2a52bb4ce636a9e95 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 18:38:46.292608 11403 raft_consensus.cc:686] T 00000000000000000000000000000000 P 633c13ed73414fa2a52bb4ce636a9e95 [term 1 LEADER]: Becoming Leader. State: Replica: 633c13ed73414fa2a52bb4ce636a9e95, State: Running, Role: LEADER
I20260806 18:38:46.292979 11403 consensus_queue.cc:237] T 00000000000000000000000000000000 P 633c13ed73414fa2a52bb4ce636a9e95 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "633c13ed73414fa2a52bb4ce636a9e95" member_type: VOTER last_known_addr { host: "127.19.194.62" port: 39303 } } peers { permanent_uuid: "67fb66439d324a3293b9b659938172b3" member_type: VOTER last_known_addr { host: "127.19.194.61" port: 44923 } } peers { permanent_uuid: "9bc603ee6c8946b9a7ab114f807854d3" member_type: VOTER last_known_addr { host: "127.19.194.60" port: 43081 } }
I20260806 18:38:46.294747 11403 sys_catalog.cc:455] T 00000000000000000000000000000000 P 633c13ed73414fa2a52bb4ce636a9e95 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 633c13ed73414fa2a52bb4ce636a9e95. Latest consensus state: current_term: 1 leader_uuid: "633c13ed73414fa2a52bb4ce636a9e95" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "633c13ed73414fa2a52bb4ce636a9e95" member_type: VOTER last_known_addr { host: "127.19.194.62" port: 39303 } } peers { permanent_uuid: "67fb66439d324a3293b9b659938172b3" member_type: VOTER last_known_addr { host: "127.19.194.61" port: 44923 } } peers { permanent_uuid: "9bc603ee6c8946b9a7ab114f807854d3" member_type: VOTER last_known_addr { host: "127.19.194.60" port: 43081 } } }
I20260806 18:38:46.294920 11403 sys_catalog.cc:458] T 00000000000000000000000000000000 P 633c13ed73414fa2a52bb4ce636a9e95 [sys.catalog]: This master's current role is: LEADER
I20260806 18:38:46.295908 11128 master_runner.cc:263] No leader master found from master 633c13ed73414fa2a52bb4ce636a9e95
I20260806 18:38:46.295980 11128 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260806 18:38:46.307117 11424 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 18:38:46.308318 11424 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 18:38:46.313548 11335 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 9bc603ee6c8946b9a7ab114f807854d3 [term 1 FOLLOWER]: Refusing update from remote peer 633c13ed73414fa2a52bb4ce636a9e95: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 18:38:46.314298 11421 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 633c13ed73414fa2a52bb4ce636a9e95 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9bc603ee6c8946b9a7ab114f807854d3" member_type: VOTER last_known_addr { host: "127.19.194.60" port: 43081 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 18:38:46.318428 11372 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9bc603ee6c8946b9a7ab114f807854d3 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 633c13ed73414fa2a52bb4ce636a9e95. Latest consensus state: current_term: 1 leader_uuid: "633c13ed73414fa2a52bb4ce636a9e95" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "633c13ed73414fa2a52bb4ce636a9e95" member_type: VOTER last_known_addr { host: "127.19.194.62" port: 39303 } } peers { permanent_uuid: "67fb66439d324a3293b9b659938172b3" member_type: VOTER last_known_addr { host: "127.19.194.61" port: 44923 } } peers { permanent_uuid: "9bc603ee6c8946b9a7ab114f807854d3" member_type: VOTER last_known_addr { host: "127.19.194.60" port: 43081 } } }
I20260806 18:38:46.318573 11372 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9bc603ee6c8946b9a7ab114f807854d3 [sys.catalog]: This master's current role is: FOLLOWER
I20260806 18:38:46.319705 11256 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 67fb66439d324a3293b9b659938172b3 [term 1 FOLLOWER]: Refusing update from remote peer 633c13ed73414fa2a52bb4ce636a9e95: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 18:38:46.320303 11370 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:38:46.299513 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43516 (local address 127.19.194.60:43081)
0806 18:38:46.299647 (+ 134us) server_negotiation.cc:206] Beginning negotiation
0806 18:38:46.299651 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 18:38:46.300732 (+ 1081us) server_negotiation.cc:407] Connection header received
0806 18:38:46.300932 (+ 200us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:38:46.300936 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:38:46.300986 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:38:46.301037 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:38:46.302045 (+ 1008us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:38:46.302860 (+ 815us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:38:46.304481 (+ 1621us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:38:46.304857 (+ 376us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:38:46.304952 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:38:46.306856 (+ 1904us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:38:46.306861 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:38:46.306864 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 18:38:46.306891 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:38:46.308681 (+ 1790us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:38:46.309172 (+ 491us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:38:46.309175 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:38:46.309177 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:38:46.309259 (+ 82us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:38:46.319274 (+ 10015us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:38:46.319278 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:38:46.319281 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:38:46.319531 (+ 250us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:38:46.319636 (+ 105us) server_negotiation.cc:1035] Waiting for connection context
0806 18:38:46.320009 (+ 373us) server_negotiation.cc:299] Negotiation successful
0806 18:38:46.320143 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260806 18:38:46.320632 11421 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 633c13ed73414fa2a52bb4ce636a9e95 [LEADER]: Connected to new peer: Peer: permanent_uuid: "67fb66439d324a3293b9b659938172b3" member_type: VOTER last_known_addr { host: "127.19.194.61" port: 44923 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 18:38:46.325454 11388 sys_catalog.cc:455] T 00000000000000000000000000000000 P 67fb66439d324a3293b9b659938172b3 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 633c13ed73414fa2a52bb4ce636a9e95. Latest consensus state: current_term: 1 leader_uuid: "633c13ed73414fa2a52bb4ce636a9e95" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "633c13ed73414fa2a52bb4ce636a9e95" member_type: VOTER last_known_addr { host: "127.19.194.62" port: 39303 } } peers { permanent_uuid: "67fb66439d324a3293b9b659938172b3" member_type: VOTER last_known_addr { host: "127.19.194.61" port: 44923 } } peers { permanent_uuid: "9bc603ee6c8946b9a7ab114f807854d3" member_type: VOTER last_known_addr { host: "127.19.194.60" port: 43081 } } }
I20260806 18:38:46.325598 11388 sys_catalog.cc:458] T 00000000000000000000000000000000 P 67fb66439d324a3293b9b659938172b3 [sys.catalog]: This master's current role is: FOLLOWER
I20260806 18:38:46.330456 11288 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:38:46.299536 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48830 (local address 127.19.194.61:44923)
0806 18:38:46.299658 (+ 122us) server_negotiation.cc:206] Beginning negotiation
0806 18:38:46.299663 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 18:38:46.304234 (+ 4571us) server_negotiation.cc:407] Connection header received
0806 18:38:46.304373 (+ 139us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:38:46.304376 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:38:46.304427 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:38:46.304498 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:38:46.305661 (+ 1163us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:38:46.306359 (+ 698us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:38:46.319272 (+ 12913us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:38:46.319601 (+ 329us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:38:46.319684 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:38:46.323253 (+ 3569us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:38:46.323257 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:38:46.323260 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 18:38:46.323306 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:38:46.325400 (+ 2094us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:38:46.326618 (+ 1218us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:38:46.326622 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:38:46.326623 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:38:46.326709 (+ 86us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:38:46.327047 (+ 338us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:38:46.327050 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:38:46.327052 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:38:46.327297 (+ 245us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:38:46.327524 (+ 227us) server_negotiation.cc:1035] Waiting for connection context
0806 18:38:46.330163 (+ 2639us) server_negotiation.cc:299] Negotiation successful
0806 18:38:46.330291 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260806 18:38:46.331729 11421 sys_catalog.cc:455] T 00000000000000000000000000000000 P 633c13ed73414fa2a52bb4ce636a9e95 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "633c13ed73414fa2a52bb4ce636a9e95" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "633c13ed73414fa2a52bb4ce636a9e95" member_type: VOTER last_known_addr { host: "127.19.194.62" port: 39303 } } peers { permanent_uuid: "67fb66439d324a3293b9b659938172b3" member_type: VOTER last_known_addr { host: "127.19.194.61" port: 44923 } } peers { permanent_uuid: "9bc603ee6c8946b9a7ab114f807854d3" member_type: VOTER last_known_addr { host: "127.19.194.60" port: 43081 } } }
I20260806 18:38:46.331861 11421 sys_catalog.cc:458] T 00000000000000000000000000000000 P 633c13ed73414fa2a52bb4ce636a9e95 [sys.catalog]: This master's current role is: LEADER
I20260806 18:38:46.333317 11388 sys_catalog.cc:455] T 00000000000000000000000000000000 P 67fb66439d324a3293b9b659938172b3 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "633c13ed73414fa2a52bb4ce636a9e95" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "633c13ed73414fa2a52bb4ce636a9e95" member_type: VOTER last_known_addr { host: "127.19.194.62" port: 39303 } } peers { permanent_uuid: "67fb66439d324a3293b9b659938172b3" member_type: VOTER last_known_addr { host: "127.19.194.61" port: 44923 } } peers { permanent_uuid: "9bc603ee6c8946b9a7ab114f807854d3" member_type: VOTER last_known_addr { host: "127.19.194.60" port: 43081 } } }
I20260806 18:38:46.333420 11368 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:38:46.300132 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40574 (local address 127.19.194.62:39303)
0806 18:38:46.300243 (+ 111us) server_negotiation.cc:206] Beginning negotiation
0806 18:38:46.300247 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 18:38:46.305590 (+ 5343us) server_negotiation.cc:407] Connection header received
0806 18:38:46.307572 (+ 1982us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:38:46.307575 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:38:46.307619 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:38:46.307668 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:38:46.308521 (+ 853us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:38:46.309223 (+ 702us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:38:46.321649 (+ 12426us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:38:46.322003 (+ 354us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:38:46.322085 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:38:46.327855 (+ 5770us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:38:46.327859 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:38:46.327862 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 18:38:46.327890 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:38:46.329602 (+ 1712us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:38:46.331899 (+ 2297us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:38:46.331903 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:38:46.331904 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:38:46.331989 (+ 85us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:38:46.332364 (+ 375us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:38:46.332367 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:38:46.332369 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:38:46.332627 (+ 258us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:38:46.332755 (+ 128us) server_negotiation.cc:1035] Waiting for connection context
0806 18:38:46.333122 (+ 367us) server_negotiation.cc:299] Negotiation successful
0806 18:38:46.333251 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260806 18:38:46.333427 11388 sys_catalog.cc:458] T 00000000000000000000000000000000 P 67fb66439d324a3293b9b659938172b3 [sys.catalog]: This master's current role is: FOLLOWER
I20260806 18:38:46.334239 11428 mvcc.cc:205] Tried to move back new op lower bound from 7315626091773124608 to 7315626091699748864. Current Snapshot: MvccSnapshot[applied={T|T < 7315626091773124608}]
I20260806 18:38:46.336268 11431 mvcc.cc:205] Tried to move back new op lower bound from 7315626091773124608 to 7315626091699748864. Current Snapshot: MvccSnapshot[applied={T|T < 7315626091773124608}]
I20260806 18:38:46.336956 11403 sys_catalog.cc:455] T 00000000000000000000000000000000 P 633c13ed73414fa2a52bb4ce636a9e95 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "633c13ed73414fa2a52bb4ce636a9e95" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "633c13ed73414fa2a52bb4ce636a9e95" member_type: VOTER last_known_addr { host: "127.19.194.62" port: 39303 } } peers { permanent_uuid: "67fb66439d324a3293b9b659938172b3" member_type: VOTER last_known_addr { host: "127.19.194.61" port: 44923 } } peers { permanent_uuid: "9bc603ee6c8946b9a7ab114f807854d3" member_type: VOTER last_known_addr { host: "127.19.194.60" port: 43081 } } }
I20260806 18:38:46.337062 11403 sys_catalog.cc:458] T 00000000000000000000000000000000 P 633c13ed73414fa2a52bb4ce636a9e95 [sys.catalog]: This master's current role is: LEADER
I20260806 18:38:46.338256 11372 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9bc603ee6c8946b9a7ab114f807854d3 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "633c13ed73414fa2a52bb4ce636a9e95" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "633c13ed73414fa2a52bb4ce636a9e95" member_type: VOTER last_known_addr { host: "127.19.194.62" port: 39303 } } peers { permanent_uuid: "67fb66439d324a3293b9b659938172b3" member_type: VOTER last_known_addr { host: "127.19.194.61" port: 44923 } } peers { permanent_uuid: "9bc603ee6c8946b9a7ab114f807854d3" member_type: VOTER last_known_addr { host: "127.19.194.60" port: 43081 } } }
I20260806 18:38:46.338370 11372 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9bc603ee6c8946b9a7ab114f807854d3 [sys.catalog]: This master's current role is: FOLLOWER
I20260806 18:38:46.339680 11429 mvcc.cc:205] Tried to move back new op lower bound from 7315626091773124608 to 7315626091699748864. Current Snapshot: MvccSnapshot[applied={T|T < 7315626091773124608}]
I20260806 18:38:46.340760 11424 catalog_manager.cc:1347] Generated new cluster ID: a4b5043bc5aa467d94542853321e20d1
I20260806 18:38:46.340824 11424 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 18:38:46.374145 11424 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 18:38:46.375536 11424 catalog_manager.cc:1504] Loading token signing keys...
I20260806 18:38:46.386880 11424 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 633c13ed73414fa2a52bb4ce636a9e95: Generated new TSK 0
I20260806 18:38:46.387699 11424 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260806 18:38:46.421240 11370 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 18:38:46.418776 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43528 (local address 127.19.194.60:43081)
0806 18:38:46.418899 (+ 123us) server_negotiation.cc:206] Beginning negotiation
0806 18:38:46.418904 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 18:38:46.420179 (+ 1275us) server_negotiation.cc:407] Connection header received
0806 18:38:46.420393 (+ 214us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:38:46.420396 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:38:46.420443 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:38:46.420496 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:38:46.421102 (+ 606us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:43528: BlockingRecv error: recv got EOF from 127.0.0.1:43528 (error 108)
Metrics: {"server-negotiator.queue_time_us":47}
W20260806 18:38:46.422576 11288 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 18:38:46.419587 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48834 (local address 127.19.194.61:44923)
0806 18:38:46.419694 (+ 107us) server_negotiation.cc:206] Beginning negotiation
0806 18:38:46.419698 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 18:38:46.421680 (+ 1982us) server_negotiation.cc:407] Connection header received
0806 18:38:46.421854 (+ 174us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:38:46.421859 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:38:46.421906 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:38:46.421956 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:38:46.422459 (+ 503us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:48834: BlockingRecv error: recv got EOF from 127.0.0.1:48834 (error 108)
Metrics: {"server-negotiator.queue_time_us":43}
W20260806 18:38:46.424551 11368 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 18:38:46.423225 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40580 (local address 127.19.194.62:39303)
0806 18:38:46.423350 (+ 125us) server_negotiation.cc:206] Beginning negotiation
0806 18:38:46.423354 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 18:38:46.423664 (+ 310us) server_negotiation.cc:407] Connection header received
0806 18:38:46.423860 (+ 196us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:38:46.423863 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:38:46.423909 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:38:46.423960 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:38:46.424443 (+ 483us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:40580: BlockingRecv error: recv got EOF from 127.0.0.1:40580 (error 108)
Metrics: {"server-negotiator.queue_time_us":49}
W20260806 18:38:46.428925 11288 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 18:38:46.427449 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48836 (local address 127.19.194.61:44923)
0806 18:38:46.427545 (+ 96us) server_negotiation.cc:206] Beginning negotiation
0806 18:38:46.427549 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 18:38:46.427983 (+ 434us) server_negotiation.cc:407] Connection header received
0806 18:38:46.428188 (+ 205us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:38:46.428193 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:38:46.428265 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:38:46.428330 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:38:46.428824 (+ 494us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:48836: BlockingRecv error: recv got EOF from 127.0.0.1:48836 (error 108)
Metrics: {"server-negotiator.queue_time_us":36}
W20260806 18:38:46.433841 11288 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 18:38:46.432438 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48848 (local address 127.19.194.61:44923)
0806 18:38:46.432530 (+ 92us) server_negotiation.cc:206] Beginning negotiation
0806 18:38:46.432534 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 18:38:46.432977 (+ 443us) server_negotiation.cc:407] Connection header received
0806 18:38:46.433147 (+ 170us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:38:46.433150 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:38:46.433196 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:38:46.433241 (+ 45us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:38:46.433751 (+ 510us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:48848: BlockingRecv error: recv got EOF from 127.0.0.1:48848 (error 108)
Metrics: {"server-negotiator.queue_time_us":37}
W20260806 18:38:46.439426 11288 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 18:38:46.437908 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48850 (local address 127.19.194.61:44923)
0806 18:38:46.437997 (+ 89us) server_negotiation.cc:206] Beginning negotiation
0806 18:38:46.438001 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 18:38:46.438525 (+ 524us) server_negotiation.cc:407] Connection header received
0806 18:38:46.438697 (+ 172us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:38:46.438700 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:38:46.438748 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:38:46.438796 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:38:46.439327 (+ 531us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:48850: BlockingRecv error: recv got EOF from 127.0.0.1:48850 (error 108)
Metrics: {"server-negotiator.queue_time_us":36}
W20260806 18:38:46.447037 11288 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 18:38:46.445507 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48866 (local address 127.19.194.61:44923)
0806 18:38:46.445633 (+ 126us) server_negotiation.cc:206] Beginning negotiation
0806 18:38:46.445637 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 18:38:46.446123 (+ 486us) server_negotiation.cc:407] Connection header received
0806 18:38:46.446350 (+ 227us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:38:46.446354 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:38:46.446402 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:38:46.446455 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:38:46.446948 (+ 493us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:48866: BlockingRecv error: recv got EOF from 127.0.0.1:48866 (error 108)
Metrics: {"server-negotiator.queue_time_us":68}
W20260806 18:38:46.459123 11288 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 18:38:46.457413 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48872 (local address 127.19.194.61:44923)
0806 18:38:46.457515 (+ 102us) server_negotiation.cc:206] Beginning negotiation
0806 18:38:46.457519 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 18:38:46.458076 (+ 557us) server_negotiation.cc:407] Connection header received
0806 18:38:46.458402 (+ 326us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:38:46.458406 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:38:46.458457 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:38:46.458509 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:38:46.459015 (+ 506us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:48872: BlockingRecv error: recv got EOF from 127.0.0.1:48872 (error 108)
Metrics: {"server-negotiator.queue_time_us":41}
W20260806 18:38:46.479422 11288 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 18:38:46.477572 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48886 (local address 127.19.194.61:44923)
0806 18:38:46.477684 (+ 112us) server_negotiation.cc:206] Beginning negotiation
0806 18:38:46.477688 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 18:38:46.478241 (+ 553us) server_negotiation.cc:407] Connection header received
0806 18:38:46.478553 (+ 312us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:38:46.478557 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:38:46.478626 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:38:46.478681 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:38:46.479305 (+ 624us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:48886: BlockingRecv error: recv got EOF from 127.0.0.1:48886 (error 108)
Metrics: {"server-negotiator.queue_time_us":41}
W20260806 18:38:46.515372 11288 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 18:38:46.513954 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48898 (local address 127.19.194.61:44923)
0806 18:38:46.514104 (+ 150us) server_negotiation.cc:206] Beginning negotiation
0806 18:38:46.514110 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 18:38:46.514435 (+ 325us) server_negotiation.cc:407] Connection header received
0806 18:38:46.514614 (+ 179us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:38:46.514617 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:38:46.514686 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:38:46.514774 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:38:46.515255 (+ 481us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:48898: BlockingRecv error: recv got EOF from 127.0.0.1:48898 (error 108)
Metrics: {"server-negotiator.queue_time_us":57}
W20260806 18:38:46.582645 11288 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 18:38:46.581478 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48914 (local address 127.19.194.61:44923)
0806 18:38:46.581630 (+ 152us) server_negotiation.cc:206] Beginning negotiation
0806 18:38:46.581633 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 18:38:46.581806 (+ 173us) server_negotiation.cc:407] Connection header received
0806 18:38:46.581937 (+ 131us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:38:46.581940 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:38:46.581990 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:38:46.582084 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:38:46.582514 (+ 430us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:48914: BlockingRecv error: recv got EOF from 127.0.0.1:48914 (error 108)
Metrics: {"server-negotiator.queue_time_us":63}
W20260806 18:38:46.714087 11288 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 18:38:46.712996 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48916 (local address 127.19.194.61:44923)
0806 18:38:46.713145 (+ 149us) server_negotiation.cc:206] Beginning negotiation
0806 18:38:46.713149 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 18:38:46.713287 (+ 138us) server_negotiation.cc:407] Connection header received
0806 18:38:46.713416 (+ 129us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:38:46.713419 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:38:46.713473 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:38:46.713555 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:38:46.713984 (+ 429us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:48916: BlockingRecv error: recv got EOF from 127.0.0.1:48916 (error 108)
Metrics: {"server-negotiator.queue_time_us":54}
W20260806 18:38:46.973588 11288 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 18:38:46.972467 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48920 (local address 127.19.194.61:44923)
0806 18:38:46.972608 (+ 141us) server_negotiation.cc:206] Beginning negotiation
0806 18:38:46.972616 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 18:38:46.972710 (+ 94us) server_negotiation.cc:407] Connection header received
0806 18:38:46.972893 (+ 183us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:38:46.972897 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:38:46.972955 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:38:46.973051 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:38:46.973481 (+ 430us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:48920: BlockingRecv error: recv got EOF from 127.0.0.1:48920 (error 108)
Metrics: {"server-negotiator.queue_time_us":53}
I20260806 18:38:47.268805 11384 catalog_manager.cc:1259] Loaded cluster ID: a4b5043bc5aa467d94542853321e20d1
I20260806 18:38:47.268893 11384 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 9bc603ee6c8946b9a7ab114f807854d3: loading cluster ID for follower catalog manager: success
I20260806 18:38:47.270650 11384 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 9bc603ee6c8946b9a7ab114f807854d3: acquiring CA information for follower catalog manager: success
I20260806 18:38:47.271879 11384 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 9bc603ee6c8946b9a7ab114f807854d3: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260806 18:38:47.281876 11400 catalog_manager.cc:1259] Loaded cluster ID: a4b5043bc5aa467d94542853321e20d1
I20260806 18:38:47.281939 11400 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 67fb66439d324a3293b9b659938172b3: loading cluster ID for follower catalog manager: success
I20260806 18:38:47.283627 11400 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 67fb66439d324a3293b9b659938172b3: acquiring CA information for follower catalog manager: success
I20260806 18:38:47.284735 11400 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 67fb66439d324a3293b9b659938172b3: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260806 18:38:47.491487 11502 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:38:47.487894 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48924 (local address 127.19.194.61:44923)
0806 18:38:47.488110 (+ 216us) server_negotiation.cc:206] Beginning negotiation
0806 18:38:47.488114 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 18:38:47.488196 (+ 82us) server_negotiation.cc:407] Connection header received
0806 18:38:47.488316 (+ 120us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:38:47.488319 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:38:47.488452 (+ 133us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:38:47.488559 (+ 107us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 18:38:47.488788 (+ 229us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:38:47.489285 (+ 497us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:38:47.490072 (+ 787us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:38:47.490280 (+ 208us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:38:47.490352 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:38:47.490616 (+ 264us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 18:38:47.491030 (+ 414us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 18:38:47.491168 (+ 138us) server_negotiation.cc:1035] Waiting for connection context
0806 18:38:47.491237 (+ 69us) server_negotiation.cc:299] Negotiation successful
0806 18:38:47.491295 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":157,"thread_start_us":64,"threads_started":1}
I20260806 18:38:47.497716 11508 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:38:47.494383 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43536 (local address 127.19.194.60:43081)
0806 18:38:47.494601 (+ 218us) server_negotiation.cc:206] Beginning negotiation
0806 18:38:47.494605 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 18:38:47.494618 (+ 13us) server_negotiation.cc:407] Connection header received
0806 18:38:47.494798 (+ 180us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:38:47.494801 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:38:47.494908 (+ 107us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:38:47.495004 (+ 96us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 18:38:47.495236 (+ 232us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:38:47.495761 (+ 525us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:38:47.496490 (+ 729us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:38:47.496715 (+ 225us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:38:47.496787 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:38:47.497048 (+ 261us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 18:38:47.497396 (+ 348us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 18:38:47.497484 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0806 18:38:47.497541 (+ 57us) server_negotiation.cc:299] Negotiation successful
0806 18:38:47.497586 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":149,"thread_start_us":84,"threads_started":1}
I20260806 18:38:47.504477 11502 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:38:47.500779 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48930 (local address 127.19.194.61:44923)
0806 18:38:47.500896 (+ 117us) server_negotiation.cc:206] Beginning negotiation
0806 18:38:47.500900 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 18:38:47.501035 (+ 135us) server_negotiation.cc:407] Connection header received
0806 18:38:47.501151 (+ 116us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:38:47.501153 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:38:47.501201 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:38:47.501305 (+ 104us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 18:38:47.501556 (+ 251us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:38:47.502389 (+ 833us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:38:47.503383 (+ 994us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:38:47.503604 (+ 221us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:38:47.503717 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:38:47.504032 (+ 315us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 18:38:47.504095 (+ 63us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 18:38:47.504151 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0806 18:38:47.504272 (+ 121us) server_negotiation.cc:299] Negotiation successful
0806 18:38:47.504339 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260806 18:38:47.505338 11508 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:38:47.500606 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43552 (local address 127.19.194.60:43081)
0806 18:38:47.500718 (+ 112us) server_negotiation.cc:206] Beginning negotiation
0806 18:38:47.500724 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 18:38:47.501085 (+ 361us) server_negotiation.cc:407] Connection header received
0806 18:38:47.501243 (+ 158us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:38:47.501246 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:38:47.501332 (+ 86us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:38:47.501399 (+ 67us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 18:38:47.501808 (+ 409us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:38:47.502593 (+ 785us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:38:47.503944 (+ 1351us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:38:47.504238 (+ 294us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:38:47.504339 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:38:47.504615 (+ 276us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 18:38:47.504693 (+ 78us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 18:38:47.504767 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0806 18:38:47.505169 (+ 402us) server_negotiation.cc:299] Negotiation successful
0806 18:38:47.505219 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34}
I20260806 18:38:47.506143 11515 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:38:47.500482 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40586 (local address 127.19.194.62:39303)
0806 18:38:47.501316 (+ 834us) server_negotiation.cc:206] Beginning negotiation
0806 18:38:47.501320 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 18:38:47.501333 (+ 13us) server_negotiation.cc:407] Connection header received
0806 18:38:47.501416 (+ 83us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:38:47.501419 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:38:47.501568 (+ 149us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:38:47.501679 (+ 111us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 18:38:47.501947 (+ 268us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:38:47.502576 (+ 629us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:38:47.504424 (+ 1848us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:38:47.504788 (+ 364us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:38:47.504884 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:38:47.505211 (+ 327us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 18:38:47.505721 (+ 510us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 18:38:47.505824 (+ 103us) server_negotiation.cc:1035] Waiting for connection context
0806 18:38:47.505942 (+ 118us) server_negotiation.cc:299] Negotiation successful
0806 18:38:47.506001 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":715,"thread_start_us":68,"threads_started":1}
I20260806 18:38:47.513729 20232 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJW69tn/build/debug/bin/kudu with pid 11128
I20260806 18:38:47.522250 20232 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJW69tn/build/debug/bin/kudu with pid 11208
I20260806 18:38:47.528807 20232 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJW69tn/build/debug/bin/kudu with pid 11291
2026-08-06T18:38:47Z chronyd exiting
[ OK ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/0 (3088 ms)
[ RUN ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/1
Loading random data
Initializing database '/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786041246094603-20232-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 18:38:47 dist-test-slave-1xrh krb5kdc[11520](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 18:38:47 dist-test-slave-1xrh krb5kdc[11520](info): set up 2 sockets
Aug 06 18:38:47 dist-test-slave-1xrh krb5kdc[11520](info): commencing operation
krb5kdc: starting...
W20260806 18:38:48.588370 20232 mini_kdc.cc:121] Time spent starting KDC: real 1.037s user 0.002s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786041246094603-20232-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786041246094603-20232-0/krb5kdc/test-user.keytab.
Aug 06 18:38:48 dist-test-slave-1xrh krb5kdc[11520](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041528, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T18:38:48Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T18:38:48Z Disabled control of system clock
WARNING: no policy specified for kudu/127.19.194.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.19.194.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.19.194.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786041246094603-20232-0/krb5kdc/kudu_127.19.194.62.keytab.
Entry for principal kudu/127.19.194.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786041246094603-20232-0/krb5kdc/kudu_127.19.194.62.keytab.
I20260806 18:38:48.707077 20232 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJW69tn/build/debug/bin/kudu
/tmp/dist-test-taskJW69tn/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786041246094603-20232-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786041246094603-20232-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786041246094603-20232-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786041246094603-20232-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.19.194.62:43495
--webserver_interface=127.19.194.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786041246094603-20232-0/krb5kdc/kudu_127.19.194.62.keytab
--principal=kudu/127.19.194.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.19.194.20:36765
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.19.194.62:43495,127.19.194.61:36371,127.19.194.60:44191
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786041246094603-20232-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786041246094603-20232-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786041246094603-20232-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 18:38:48.813761 11535 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260806 18:38:48.814025 11535 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 18:38:48.814047 11535 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 18:38:48.814085 11535 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 18:38:48.817932 11535 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 18:38:48.817992 11535 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 18:38:48.818010 11535 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 18:38:48.818027 11535 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 18:38:48.818040 11535 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 18:38:48.823009 11535 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.194.20:36765
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786041246094603-20232-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786041246094603-20232-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.19.194.62:43495,127.19.194.61:36371,127.19.194.60:44191
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786041246094603-20232-0/krb5kdc/kudu_127.19.194.62.keytab
--principal=kudu/127.19.194.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.19.194.62:43495
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786041246094603-20232-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.19.194.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11535
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786041246094603-20232-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 17:59:13 UTC on ccycloud.quasar-wvsnzx.root.comops.site
build id 4571028
I20260806 18:38:48.824028 11535 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 18:38:48.824939 11535 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 18:38:48.833005 11540 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:38:48.833022 11543 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:38:48.833053 11541 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 18:38:48.833372 11535 server_base.cc:1034] running on GCE node
I20260806 18:38:48.833918 11535 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 18:38:48.834955 11535 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 18:38:48.836143 11535 hybrid_clock.cc:648] HybridClock initialized: now 1786041528836132 us; error 34 us; skew 500 ppm
Aug 06 18:38:48 dist-test-slave-1xrh krb5kdc[11520](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041528, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.194.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 18:38:48.838811 11535 init.cc:377] Logged in from keytab as kudu/127.19.194.62@KRBTEST.COM (short username kudu)
I20260806 18:38:48.839687 11535 webserver.cc:460] Webserver started at http://127.19.194.62:37213/ using document root <none> and password file <none>
I20260806 18:38:48.840212 11535 fs_manager.cc:359] Metadata directory not provided
I20260806 18:38:48.840308 11535 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 18:38:48.840583 11535 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 18:38:48.842669 11535 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786041246094603-20232-0/minicluster-data/master-0/data/instance:
uuid: "a8d621a6deec497b88ee744ab94c452f"
format_stamp: "Formatted at 2026-08-06 18:38:48 on dist-test-slave-1xrh"
server_key: "942e6099e24f0282fe4d0294cb17e4b7"
server_key_iv: "43b7333aba2d9edd5e017ba1e425ac48"
server_key_version: "encryptionkey@0"
I20260806 18:38:48.843217 11535 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786041246094603-20232-0/minicluster-data/master-0/wal/instance:
uuid: "a8d621a6deec497b88ee744ab94c452f"
format_stamp: "Formatted at 2026-08-06 18:38:48 on dist-test-slave-1xrh"
server_key: "942e6099e24f0282fe4d0294cb17e4b7"
server_key_iv: "43b7333aba2d9edd5e017ba1e425ac48"
server_key_version: "encryptionkey@0"
I20260806 18:38:48.846675 11535 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.000s
I20260806 18:38:48.848942 11550 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 18:38:48.850099 11535 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260806 18:38:48.850224 11535 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786041246094603-20232-0/minicluster-data/master-0/data,/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786041246094603-20232-0/minicluster-data/master-0/wal
uuid: "a8d621a6deec497b88ee744ab94c452f"
format_stamp: "Formatted at 2026-08-06 18:38:48 on dist-test-slave-1xrh"
server_key: "942e6099e24f0282fe4d0294cb17e4b7"
server_key_iv: "43b7333aba2d9edd5e017ba1e425ac48"
server_key_version: "encryptionkey@0"
I20260806 18:38:48.850322 11535 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786041246094603-20232-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786041246094603-20232-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786041246094603-20232-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 18:38:48.867388 11535 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 18:38:48.870003 11535 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 18:38:48.870203 11535 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 18:38:48.877149 11535 rpc_server.cc:307] RPC server started. Bound to: 127.19.194.62:43495
I20260806 18:38:48.877429 11605 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.194.62:43495 every 8 connection(s)
I20260806 18:38:48.878084 11535 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786041246094603-20232-0/minicluster-data/master-0/data/info.pb
I20260806 18:38:48.880649 11606 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 18:38:48.882836 20232 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJW69tn/build/debug/bin/kudu as pid 11535
I20260806 18:38:48.882972 20232 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786041246094603-20232-0/minicluster-data/master-0/wal/instance
I20260806 18:38:48.883306 20232 external_mini_cluster.cc:1428] Setting key be044ab3c86528a8d46728bee13dce9d
I20260806 18:38:48.884403 11606 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.19.194.62" port: 43495 } has no permanent_uuid. Determining permanent_uuid...
WARNING: no policy specified for kudu/127.19.194.61@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.19.194.61@KRBTEST.COM" created.
Aug 06 18:38:48 dist-test-slave-1xrh krb5kdc[11520](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041528, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.194.62@KRBTEST.COM for kudu/127.19.194.62@KRBTEST.COM
I20260806 18:38:48.901341 11609 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:38:48.886766 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.194.62:43495 (local address 127.0.0.1:37222)
0806 18:38:48.887222 (+ 456us) negotiation.cc:107] Waiting for socket to connect
0806 18:38:48.887273 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0806 18:38:48.888178 (+ 905us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:38:48.888997 (+ 819us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:38:48.889002 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:38:48.889309 (+ 307us) client_negotiation.cc:189] Negotiated authn=SASL
0806 18:38:48.889590 (+ 281us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:38:48.889603 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:38:48.891285 (+ 1682us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:38:48.891287 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:38:48.892009 (+ 722us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:38:48.892017 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:38:48.892812 (+ 795us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:38:48.892815 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:38:48.892920 (+ 105us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:38:48.893494 (+ 574us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 18:38:48.893510 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0806 18:38:48.896072 (+ 2562us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 18:38:48.898461 (+ 2389us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:38:48.898470 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:38:48.898479 (+ 9us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:38:48.898709 (+ 230us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:38:48.899083 (+ 374us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:38:48.899085 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:38:48.899087 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:38:48.899256 (+ 169us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:38:48.900051 (+ 795us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 18:38:48.900056 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 18:38:48.900182 (+ 126us) client_negotiation.cc:769] Sending connection context
0806 18:38:48.900317 (+ 135us) client_negotiation.cc:240] Negotiation successful
0806 18:38:48.900565 (+ 248us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":254,"mutex_wait_us":1,"thread_start_us":139,"threads_started":1}
I20260806 18:38:48.901346 11610 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:38:48.886766 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37222 (local address 127.19.194.62:43495)
0806 18:38:48.887295 (+ 529us) server_negotiation.cc:206] Beginning negotiation
0806 18:38:48.887308 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0806 18:38:48.887373 (+ 65us) server_negotiation.cc:407] Connection header received
0806 18:38:48.888501 (+ 1128us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:38:48.888508 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:38:48.888774 (+ 266us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:38:48.888993 (+ 219us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:38:48.889737 (+ 744us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:38:48.891167 (+ 1430us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:38:48.892136 (+ 969us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:38:48.892674 (+ 538us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:38:48.892793 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:38:48.896225 (+ 3432us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:38:48.896230 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:38:48.896240 (+ 10us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 18:38:48.896261 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:38:48.898310 (+ 2049us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:38:48.898826 (+ 516us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:38:48.898836 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:38:48.898840 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:38:48.898985 (+ 145us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:38:48.899362 (+ 377us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:38:48.899368 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:38:48.899373 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:38:48.899928 (+ 555us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:38:48.900076 (+ 148us) server_negotiation.cc:1035] Waiting for connection context
0806 18:38:48.900470 (+ 394us) server_negotiation.cc:299] Negotiation successful
0806 18:38:48.900652 (+ 182us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":62,"server-negotiator.queue_time_us":404,"spinlock_wait_cycles":13184,"thread_start_us":143,"threads_started":1}
I20260806 18:38:48.904012 11606 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.19.194.61" port: 36371 } has no permanent_uuid. Determining permanent_uuid...
I20260806 18:38:48.904770 11609 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:38:48.904509 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.194.61:36371 (local address 127.0.0.1:52142)
0806 18:38:48.904635 (+ 126us) negotiation.cc:107] Waiting for socket to connect
0806 18:38:48.904712 (+ 77us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.194.61:36371: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":50}
W20260806 18:38:48.904999 11554 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.19.194.61:36371: connect: Connection refused (error 111)
I20260806 18:38:48.905843 11609 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:38:48.905629 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.194.61:36371 (local address 127.0.0.1:52150)
0806 18:38:48.905749 (+ 120us) negotiation.cc:107] Waiting for socket to connect
0806 18:38:48.905793 (+ 44us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.194.61:36371: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":46}
W20260806 18:38:48.905982 11606 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.19.194.61:36371: Network error: Client connection negotiation failed: client connection to 127.19.194.61:36371: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.19.194.61 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786041246094603-20232-0/krb5kdc/kudu_127.19.194.61.keytab.
Entry for principal kudu/127.19.194.61 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786041246094603-20232-0/krb5kdc/kudu_127.19.194.61.keytab.
I20260806 18:38:48.913532 20232 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJW69tn/build/debug/bin/kudu
/tmp/dist-test-taskJW69tn/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786041246094603-20232-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786041246094603-20232-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786041246094603-20232-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786041246094603-20232-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.19.194.61:36371
--webserver_interface=127.19.194.61
--webserver_port=0
--keytab_file=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786041246094603-20232-0/krb5kdc/kudu_127.19.194.61.keytab
--principal=kudu/127.19.194.61
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.19.194.20:36765
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.19.194.62:43495,127.19.194.61:36371,127.19.194.60:44191
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786041246094603-20232-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786041246094603-20232-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786041246094603-20232-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260806 18:38:48.955204 11606 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.19.194.61" port: 36371 } attempt: 1
I20260806 18:38:48.956070 11609 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:38:48.955761 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.194.61:36371 (local address 127.0.0.1:52166)
0806 18:38:48.955927 (+ 166us) negotiation.cc:107] Waiting for socket to connect
0806 18:38:48.955990 (+ 63us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.194.61:36371: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":61}
I20260806 18:38:48.956815 11609 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:38:48.956625 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.194.61:36371 (local address 127.0.0.1:52180)
0806 18:38:48.956719 (+ 94us) negotiation.cc:107] Waiting for socket to connect
0806 18:38:48.956762 (+ 43us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.194.61:36371: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":38}
W20260806 18:38:48.956975 11606 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.19.194.61:36371: Network error: Client connection negotiation failed: client connection to 127.19.194.61:36371: connect: Connection refused (error 111)
W20260806 18:38:49.022836 11613 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260806 18:38:49.023069 11613 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 18:38:49.023119 11613 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 18:38:49.023208 11613 flags.cc:425] Enabled unsafe flag: --never_fsync=true
I20260806 18:38:49.025144 11606 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.19.194.61" port: 36371 } attempt: 2
I20260806 18:38:49.025961 11609 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:38:49.025675 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.194.61:36371 (local address 127.0.0.1:52182)
0806 18:38:49.025820 (+ 145us) negotiation.cc:107] Waiting for socket to connect
0806 18:38:49.025880 (+ 60us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.194.61:36371: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":57}
I20260806 18:38:49.026722 11609 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:38:49.026537 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.194.61:36371 (local address 127.0.0.1:52192)
0806 18:38:49.026630 (+ 93us) negotiation.cc:107] Waiting for socket to connect
0806 18:38:49.026672 (+ 42us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.194.61:36371: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":38}
W20260806 18:38:49.026872 11606 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.19.194.61:36371: Network error: Client connection negotiation failed: client connection to 127.19.194.61:36371: connect: Connection refused (error 111)
W20260806 18:38:49.027823 11613 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 18:38:49.027882 11613 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 18:38:49.027943 11613 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 18:38:49.027978 11613 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 18:38:49.028004 11613 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 18:38:49.033136 11613 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.194.20:36765
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786041246094603-20232-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786041246094603-20232-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.19.194.62:43495,127.19.194.61:36371,127.19.194.60:44191
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786041246094603-20232-0/krb5kdc/kudu_127.19.194.61.keytab
--principal=kudu/127.19.194.61
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.19.194.61:36371
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786041246094603-20232-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.19.194.61
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11613
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786041246094603-20232-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 17:59:13 UTC on ccycloud.quasar-wvsnzx.root.comops.site
build id 4571028
I20260806 18:38:49.034112 11613 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 18:38:49.035068 11613 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260806 18:38:49.043222 11613 server_base.cc:1034] running on GCE node
W20260806 18:38:49.043180 11618 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:38:49.043156 11619 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:38:49.043171 11621 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 18:38:49.043908 11613 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 18:38:49.044965 11613 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 18:38:49.046136 11613 hybrid_clock.cc:648] HybridClock initialized: now 1786041529046116 us; error 29 us; skew 500 ppm
Aug 06 18:38:49 dist-test-slave-1xrh krb5kdc[11520](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041529, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.194.61@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 18:38:49.048833 11613 init.cc:377] Logged in from keytab as kudu/127.19.194.61@KRBTEST.COM (short username kudu)
I20260806 18:38:49.049710 11613 webserver.cc:460] Webserver started at http://127.19.194.61:42987/ using document root <none> and password file <none>
I20260806 18:38:49.050227 11613 fs_manager.cc:359] Metadata directory not provided
I20260806 18:38:49.050318 11613 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 18:38:49.050567 11613 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 18:38:49.052603 11613 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786041246094603-20232-0/minicluster-data/master-1/data/instance:
uuid: "d874d0e565ce446facb0b9810832d15e"
format_stamp: "Formatted at 2026-08-06 18:38:49 on dist-test-slave-1xrh"
server_key: "061dc1ffeca625edb044002f17788c46"
server_key_iv: "62d05ef309f9ad3e9e8c70a1612ef183"
server_key_version: "encryptionkey@0"
I20260806 18:38:49.053151 11613 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786041246094603-20232-0/minicluster-data/master-1/wal/instance:
uuid: "d874d0e565ce446facb0b9810832d15e"
format_stamp: "Formatted at 2026-08-06 18:38:49 on dist-test-slave-1xrh"
server_key: "061dc1ffeca625edb044002f17788c46"
server_key_iv: "62d05ef309f9ad3e9e8c70a1612ef183"
server_key_version: "encryptionkey@0"
I20260806 18:38:49.056794 11613 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.005s sys 0.000s
I20260806 18:38:49.059001 11628 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 18:38:49.060215 11613 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260806 18:38:49.060364 11613 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786041246094603-20232-0/minicluster-data/master-1/data,/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786041246094603-20232-0/minicluster-data/master-1/wal
uuid: "d874d0e565ce446facb0b9810832d15e"
format_stamp: "Formatted at 2026-08-06 18:38:49 on dist-test-slave-1xrh"
server_key: "061dc1ffeca625edb044002f17788c46"
server_key_iv: "62d05ef309f9ad3e9e8c70a1612ef183"
server_key_version: "encryptionkey@0"
I20260806 18:38:49.060485 11613 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786041246094603-20232-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786041246094603-20232-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786041246094603-20232-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 18:38:49.077996 11613 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 18:38:49.080716 11613 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 18:38:49.080910 11613 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 18:38:49.087949 11613 rpc_server.cc:307] RPC server started. Bound to: 127.19.194.61:36371
I20260806 18:38:49.088258 11682 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.194.61:36371 every 8 connection(s)
I20260806 18:38:49.088860 11613 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786041246094603-20232-0/minicluster-data/master-1/data/info.pb
I20260806 18:38:49.090298 20232 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJW69tn/build/debug/bin/kudu as pid 11613
I20260806 18:38:49.090396 20232 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786041246094603-20232-0/minicluster-data/master-1/wal/instance
I20260806 18:38:49.090713 20232 external_mini_cluster.cc:1428] Setting key 2c37ebd5c68c0fc79a6e2a053d52a66c
I20260806 18:38:49.091609 11683 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 18:38:49.096840 11683 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.19.194.62" port: 43495 } has no permanent_uuid. Determining permanent_uuid...
WARNING: no policy specified for kudu/127.19.194.60@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.19.194.60@KRBTEST.COM" created.
Aug 06 18:38:49 dist-test-slave-1xrh krb5kdc[11520](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041529, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.194.61@KRBTEST.COM for kudu/127.19.194.62@KRBTEST.COM
I20260806 18:38:49.112499 11610 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:38:49.099208 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37230 (local address 127.19.194.62:43495)
0806 18:38:49.099379 (+ 171us) server_negotiation.cc:206] Beginning negotiation
0806 18:38:49.099387 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 18:38:49.100172 (+ 785us) server_negotiation.cc:407] Connection header received
0806 18:38:49.101144 (+ 972us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:38:49.101147 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:38:49.101193 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:38:49.101270 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:38:49.102496 (+ 1226us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:38:49.103360 (+ 864us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:38:49.104503 (+ 1143us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:38:49.104753 (+ 250us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:38:49.104862 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:38:49.107701 (+ 2839us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:38:49.107706 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:38:49.107711 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 18:38:49.107749 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:38:49.110403 (+ 2654us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:38:49.110897 (+ 494us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:38:49.110902 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:38:49.110903 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:38:49.110998 (+ 95us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:38:49.111396 (+ 398us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:38:49.111399 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:38:49.111401 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:38:49.111641 (+ 240us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:38:49.111760 (+ 119us) server_negotiation.cc:1035] Waiting for connection context
0806 18:38:49.112218 (+ 458us) server_negotiation.cc:299] Negotiation successful
0806 18:38:49.112370 (+ 152us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260806 18:38:49.113242 11686 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:38:49.099544 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.194.62:43495 (local address 127.0.0.1:37230)
0806 18:38:49.100001 (+ 457us) negotiation.cc:107] Waiting for socket to connect
0806 18:38:49.100049 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0806 18:38:49.100851 (+ 802us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:38:49.101448 (+ 597us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:38:49.101454 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:38:49.101779 (+ 325us) client_negotiation.cc:189] Negotiated authn=SASL
0806 18:38:49.102338 (+ 559us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:38:49.102355 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:38:49.103521 (+ 1166us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:38:49.103524 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:38:49.104369 (+ 845us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:38:49.104379 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:38:49.104914 (+ 535us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:38:49.104918 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:38:49.105081 (+ 163us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:38:49.105680 (+ 599us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 18:38:49.105697 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0806 18:38:49.107507 (+ 1810us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 18:38:49.110524 (+ 3017us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:38:49.110529 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:38:49.110540 (+ 11us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:38:49.110788 (+ 248us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:38:49.111106 (+ 318us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:38:49.111108 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:38:49.111110 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:38:49.111302 (+ 192us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:38:49.111755 (+ 453us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 18:38:49.111759 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 18:38:49.111990 (+ 231us) client_negotiation.cc:769] Sending connection context
0806 18:38:49.112124 (+ 134us) client_negotiation.cc:240] Negotiation successful
0806 18:38:49.112348 (+ 224us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":260,"thread_start_us":143,"threads_started":1}
I20260806 18:38:49.114648 11683 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.19.194.61" port: 36371 } has no permanent_uuid. Determining permanent_uuid...
I20260806 18:38:49.118052 11606 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.19.194.61" port: 36371 } attempt: 3
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.19.194.60 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786041246094603-20232-0/krb5kdc/kudu_127.19.194.60.keytab.
Entry for principal kudu/127.19.194.60 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786041246094603-20232-0/krb5kdc/kudu_127.19.194.60.keytab.
I20260806 18:38:49.120338 20232 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJW69tn/build/debug/bin/kudu
/tmp/dist-test-taskJW69tn/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786041246094603-20232-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786041246094603-20232-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786041246094603-20232-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786041246094603-20232-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.19.194.60:44191
--webserver_interface=127.19.194.60
--webserver_port=0
--keytab_file=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786041246094603-20232-0/krb5kdc/kudu_127.19.194.60.keytab
--principal=kudu/127.19.194.60
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.19.194.20:36765
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.19.194.62:43495,127.19.194.61:36371,127.19.194.60:44191
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786041246094603-20232-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786041246094603-20232-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786041246094603-20232-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 06 18:38:49 dist-test-slave-1xrh krb5kdc[11520](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041529, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.194.61@KRBTEST.COM for kudu/127.19.194.61@KRBTEST.COM
Aug 06 18:38:49 dist-test-slave-1xrh krb5kdc[11520](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041528, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.194.62@KRBTEST.COM for kudu/127.19.194.61@KRBTEST.COM
I20260806 18:38:49.125072 11686 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:38:49.115177 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.194.61:36371 (local address 127.0.0.1:52200)
0806 18:38:49.115297 (+ 120us) negotiation.cc:107] Waiting for socket to connect
0806 18:38:49.115313 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0806 18:38:49.115425 (+ 112us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:38:49.116041 (+ 616us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:38:49.116043 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:38:49.116172 (+ 129us) client_negotiation.cc:189] Negotiated authn=SASL
0806 18:38:49.116329 (+ 157us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:38:49.116343 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:38:49.117390 (+ 1047us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:38:49.117393 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:38:49.117987 (+ 594us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:38:49.117996 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:38:49.118861 (+ 865us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:38:49.118865 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:38:49.118977 (+ 112us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:38:49.119495 (+ 518us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 18:38:49.119509 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0806 18:38:49.120880 (+ 1371us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 18:38:49.123253 (+ 2373us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:38:49.123258 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:38:49.123261 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:38:49.123526 (+ 265us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:38:49.123891 (+ 365us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:38:49.123896 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:38:49.123901 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:38:49.123992 (+ 91us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:38:49.124538 (+ 546us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 18:38:49.124544 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 18:38:49.124645 (+ 101us) client_negotiation.cc:769] Sending connection context
0806 18:38:49.124730 (+ 85us) client_negotiation.cc:240] Negotiation successful
0806 18:38:49.124888 (+ 158us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":47}
I20260806 18:38:49.125885 11688 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:38:49.115285 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52200 (local address 127.19.194.61:36371)
0806 18:38:49.115659 (+ 374us) server_negotiation.cc:206] Beginning negotiation
0806 18:38:49.115668 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0806 18:38:49.115707 (+ 39us) server_negotiation.cc:407] Connection header received
0806 18:38:49.115754 (+ 47us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:38:49.115762 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:38:49.115926 (+ 164us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:38:49.116032 (+ 106us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:38:49.116447 (+ 415us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:38:49.117275 (+ 828us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:38:49.118112 (+ 837us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:38:49.118749 (+ 637us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:38:49.118851 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:38:49.121021 (+ 2170us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:38:49.121030 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:38:49.121051 (+ 21us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 18:38:49.121070 (+ 19us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:38:49.123097 (+ 2027us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:38:49.123670 (+ 573us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:38:49.123678 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:38:49.123683 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:38:49.123777 (+ 94us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:38:49.124092 (+ 315us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:38:49.124097 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:38:49.124102 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:38:49.124417 (+ 315us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:38:49.125386 (+ 969us) server_negotiation.cc:1035] Waiting for connection context
0806 18:38:49.125516 (+ 130us) server_negotiation.cc:299] Negotiation successful
0806 18:38:49.125685 (+ 169us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":258,"thread_start_us":174,"threads_started":1}
I20260806 18:38:49.127871 11683 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.19.194.60" port: 44191 } has no permanent_uuid. Determining permanent_uuid...
I20260806 18:38:49.128615 11686 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:38:49.128328 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.194.60:44191 (local address 127.0.0.1:35444)
0806 18:38:49.128475 (+ 147us) negotiation.cc:107] Waiting for socket to connect
0806 18:38:49.128553 (+ 78us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.194.60:44191: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":59}
I20260806 18:38:49.129055 11689 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:38:49.118530 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52204 (local address 127.19.194.61:36371)
0806 18:38:49.118817 (+ 287us) server_negotiation.cc:206] Beginning negotiation
0806 18:38:49.118827 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0806 18:38:49.118839 (+ 12us) server_negotiation.cc:407] Connection header received
0806 18:38:49.118865 (+ 26us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:38:49.118868 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:38:49.118925 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:38:49.119012 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:38:49.119539 (+ 527us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:38:49.120184 (+ 645us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:38:49.121340 (+ 1156us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:38:49.121762 (+ 422us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:38:49.121930 (+ 168us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:38:49.123927 (+ 1997us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:38:49.123931 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:38:49.123933 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 18:38:49.123958 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:38:49.126963 (+ 3005us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:38:49.127463 (+ 500us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:38:49.127468 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:38:49.127470 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:38:49.127556 (+ 86us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:38:49.128187 (+ 631us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:38:49.128189 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:38:49.128190 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:38:49.128341 (+ 151us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:38:49.128496 (+ 155us) server_negotiation.cc:1035] Waiting for connection context
0806 18:38:49.128843 (+ 347us) server_negotiation.cc:299] Negotiation successful
0806 18:38:49.128940 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":218,"thread_start_us":105,"threads_started":1}
I20260806 18:38:49.129137 11609 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:38:49.118465 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.194.61:36371 (local address 127.0.0.1:52204)
0806 18:38:49.118624 (+ 159us) negotiation.cc:107] Waiting for socket to connect
0806 18:38:49.118638 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0806 18:38:49.118713 (+ 75us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:38:49.119052 (+ 339us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:38:49.119055 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:38:49.119266 (+ 211us) client_negotiation.cc:189] Negotiated authn=SASL
0806 18:38:49.119398 (+ 132us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:38:49.119409 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:38:49.120307 (+ 898us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:38:49.120310 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:38:49.121186 (+ 876us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:38:49.121201 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:38:49.121940 (+ 739us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:38:49.121944 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:38:49.122056 (+ 112us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:38:49.122375 (+ 319us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 18:38:49.122385 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0806 18:38:49.123745 (+ 1360us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 18:38:49.127089 (+ 3344us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:38:49.127093 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:38:49.127095 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:38:49.127354 (+ 259us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:38:49.127651 (+ 297us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:38:49.127655 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:38:49.127657 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:38:49.127749 (+ 92us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:38:49.128499 (+ 750us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 18:38:49.128503 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 18:38:49.128632 (+ 129us) client_negotiation.cc:769] Sending connection context
0806 18:38:49.128752 (+ 120us) client_negotiation.cc:240] Negotiation successful
0806 18:38:49.128914 (+ 162us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":96}
W20260806 18:38:49.129809 11631 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.19.194.60:44191: connect: Connection refused (error 111)
I20260806 18:38:49.130679 11686 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:38:49.130486 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.194.60:44191 (local address 127.0.0.1:35460)
0806 18:38:49.130589 (+ 103us) negotiation.cc:107] Waiting for socket to connect
0806 18:38:49.130633 (+ 44us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.194.60:44191: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":47}
W20260806 18:38:49.130882 11683 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.19.194.60:44191: Network error: Client connection negotiation failed: client connection to 127.19.194.60:44191: connect: Connection refused (error 111)
I20260806 18:38:49.131089 11606 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.19.194.60" port: 44191 } has no permanent_uuid. Determining permanent_uuid...
I20260806 18:38:49.131811 11609 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:38:49.131599 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.194.60:44191 (local address 127.0.0.1:35472)
0806 18:38:49.131711 (+ 112us) negotiation.cc:107] Waiting for socket to connect
0806 18:38:49.131756 (+ 45us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.194.60:44191: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":50}
I20260806 18:38:49.132536 11609 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:38:49.132285 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.194.60:44191 (local address 127.0.0.1:35488)
0806 18:38:49.132435 (+ 150us) negotiation.cc:107] Waiting for socket to connect
0806 18:38:49.132486 (+ 51us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.194.60:44191: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":74}
W20260806 18:38:49.132681 11606 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.19.194.60:44191: Network error: Client connection negotiation failed: client connection to 127.19.194.60:44191: connect: Connection refused (error 111)
I20260806 18:38:49.163826 11606 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.19.194.60" port: 44191 } attempt: 1
I20260806 18:38:49.164502 11609 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:38:49.164249 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.194.60:44191 (local address 127.0.0.1:35494)
0806 18:38:49.164379 (+ 130us) negotiation.cc:107] Waiting for socket to connect
0806 18:38:49.164437 (+ 58us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.194.60:44191: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":52}
I20260806 18:38:49.165247 11609 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:38:49.165033 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.194.60:44191 (local address 127.0.0.1:35506)
0806 18:38:49.165139 (+ 106us) negotiation.cc:107] Waiting for socket to connect
0806 18:38:49.165185 (+ 46us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.194.60:44191: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":41}
W20260806 18:38:49.165372 11606 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.19.194.60:44191: Network error: Client connection negotiation failed: client connection to 127.19.194.60:44191: connect: Connection refused (error 111)
I20260806 18:38:49.180145 11683 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.19.194.60" port: 44191 } attempt: 1
I20260806 18:38:49.180907 11686 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:38:49.180634 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.194.60:44191 (local address 127.0.0.1:35508)
0806 18:38:49.180767 (+ 133us) negotiation.cc:107] Waiting for socket to connect
0806 18:38:49.180837 (+ 70us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.194.60:44191: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":54}
I20260806 18:38:49.181814 11686 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:38:49.181410 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.194.60:44191 (local address 127.0.0.1:35524)
0806 18:38:49.181692 (+ 282us) negotiation.cc:107] Waiting for socket to connect
0806 18:38:49.181755 (+ 63us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.194.60:44191: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":212}
W20260806 18:38:49.181941 11683 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.19.194.60:44191: Network error: Client connection negotiation failed: client connection to 127.19.194.60:44191: connect: Connection refused (error 111)
W20260806 18:38:49.230531 11690 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260806 18:38:49.230809 11690 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 18:38:49.230880 11690 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 18:38:49.230963 11690 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 18:38:49.234961 11690 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 18:38:49.235034 11690 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 18:38:49.235074 11690 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 18:38:49.235177 11690 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 18:38:49.235231 11690 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 18:38:49.240235 11690 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.194.20:36765
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786041246094603-20232-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786041246094603-20232-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.19.194.62:43495,127.19.194.61:36371,127.19.194.60:44191
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786041246094603-20232-0/krb5kdc/kudu_127.19.194.60.keytab
--principal=kudu/127.19.194.60
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.19.194.60:44191
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786041246094603-20232-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.19.194.60
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11690
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786041246094603-20232-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 17:59:13 UTC on ccycloud.quasar-wvsnzx.root.comops.site
build id 4571028
I20260806 18:38:49.240531 11606 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.19.194.60" port: 44191 } attempt: 2
I20260806 18:38:49.241257 11690 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 18:38:49.241287 11609 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:38:49.241023 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.194.60:44191 (local address 127.0.0.1:35526)
0806 18:38:49.241156 (+ 133us) negotiation.cc:107] Waiting for socket to connect
0806 18:38:49.241219 (+ 63us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.194.60:44191: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":53}
I20260806 18:38:49.242015 11609 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:38:49.241814 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.194.60:44191 (local address 127.0.0.1:35534)
0806 18:38:49.241925 (+ 111us) negotiation.cc:107] Waiting for socket to connect
0806 18:38:49.241966 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.194.60:44191: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":47}
W20260806 18:38:49.242206 11606 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.19.194.60:44191: Network error: Client connection negotiation failed: client connection to 127.19.194.60:44191: connect: Connection refused (error 111)
I20260806 18:38:49.242324 11690 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260806 18:38:49.250118 11683 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.19.194.60" port: 44191 } attempt: 2
W20260806 18:38:49.250274 11700 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:38:49.250264 11697 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:38:49.250334 11698 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 18:38:49.250677 11690 server_base.cc:1034] running on GCE node
I20260806 18:38:49.250834 11686 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:38:49.250580 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.194.60:44191 (local address 127.0.0.1:35542)
0806 18:38:49.250689 (+ 109us) negotiation.cc:107] Waiting for socket to connect
0806 18:38:49.250741 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.194.60:44191: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":34}
I20260806 18:38:49.251328 11690 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 18:38:49.251564 11686 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:38:49.251354 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.194.60:44191 (local address 127.0.0.1:35552)
0806 18:38:49.251464 (+ 110us) negotiation.cc:107] Waiting for socket to connect
0806 18:38:49.251511 (+ 47us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.194.60:44191: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":37}
W20260806 18:38:49.251704 11683 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.19.194.60:44191: Network error: Client connection negotiation failed: client connection to 127.19.194.60:44191: connect: Connection refused (error 111)
I20260806 18:38:49.252391 11690 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 18:38:49.253567 11690 hybrid_clock.cc:648] HybridClock initialized: now 1786041529253559 us; error 25 us; skew 500 ppm
Aug 06 18:38:49 dist-test-slave-1xrh krb5kdc[11520](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041529, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.194.60@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 18:38:49.256217 11690 init.cc:377] Logged in from keytab as kudu/127.19.194.60@KRBTEST.COM (short username kudu)
I20260806 18:38:49.257033 11690 webserver.cc:460] Webserver started at http://127.19.194.60:40555/ using document root <none> and password file <none>
I20260806 18:38:49.257540 11690 fs_manager.cc:359] Metadata directory not provided
I20260806 18:38:49.257624 11690 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 18:38:49.257843 11690 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 18:38:49.259800 11690 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786041246094603-20232-0/minicluster-data/master-2/data/instance:
uuid: "ee69134877f94c63ade3a98ddeeab9ad"
format_stamp: "Formatted at 2026-08-06 18:38:49 on dist-test-slave-1xrh"
server_key: "436e4648a3e5e7924bb434c396540ccf"
server_key_iv: "de5223b5ef973a4c515325a995c902b3"
server_key_version: "encryptionkey@0"
I20260806 18:38:49.260334 11690 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786041246094603-20232-0/minicluster-data/master-2/wal/instance:
uuid: "ee69134877f94c63ade3a98ddeeab9ad"
format_stamp: "Formatted at 2026-08-06 18:38:49 on dist-test-slave-1xrh"
server_key: "436e4648a3e5e7924bb434c396540ccf"
server_key_iv: "de5223b5ef973a4c515325a995c902b3"
server_key_version: "encryptionkey@0"
I20260806 18:38:49.263849 11690 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.001s
I20260806 18:38:49.266332 11708 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 18:38:49.267668 11690 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260806 18:38:49.267832 11690 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786041246094603-20232-0/minicluster-data/master-2/data,/tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786041246094603-20232-0/minicluster-data/master-2/wal
uuid: "ee69134877f94c63ade3a98ddeeab9ad"
format_stamp: "Formatted at 2026-08-06 18:38:49 on dist-test-slave-1xrh"
server_key: "436e4648a3e5e7924bb434c396540ccf"
server_key_iv: "de5223b5ef973a4c515325a995c902b3"
server_key_version: "encryptionkey@0"
I20260806 18:38:49.267943 11690 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786041246094603-20232-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786041246094603-20232-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786041246094603-20232-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 18:38:49.284401 11690 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 18:38:49.286922 11690 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 18:38:49.287154 11690 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 18:38:49.293727 11690 rpc_server.cc:307] RPC server started. Bound to: 127.19.194.60:44191
I20260806 18:38:49.294006 11763 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.194.60:44191 every 8 connection(s)
I20260806 18:38:49.294638 11690 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786041246094603-20232-0/minicluster-data/master-2/data/info.pb
I20260806 18:38:49.297176 11765 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 18:38:49.298002 20232 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJW69tn/build/debug/bin/kudu as pid 11690
I20260806 18:38:49.298132 20232 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJW69tn/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786041246094603-20232-0/minicluster-data/master-2/wal/instance
I20260806 18:38:49.298492 20232 external_mini_cluster.cc:1428] Setting key 69446c6289cfcdb8619e1ee9bc7e26e5
I20260806 18:38:49.301230 11765 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.19.194.62" port: 43495 } has no permanent_uuid. Determining permanent_uuid...
Aug 06 18:38:49 dist-test-slave-1xrh krb5kdc[11520](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041528, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.19.194.60@KRBTEST.COM
Aug 06 18:38:49 dist-test-slave-1xrh krb5kdc[11520](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041529, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.194.60@KRBTEST.COM for kudu/127.19.194.62@KRBTEST.COM
I20260806 18:38:49.312444 11768 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:38:49.299834 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35554 (local address 127.19.194.60:44191)
0806 18:38:49.300225 (+ 391us) server_negotiation.cc:206] Beginning negotiation
0806 18:38:49.300239 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0806 18:38:49.300272 (+ 33us) server_negotiation.cc:407] Connection header received
0806 18:38:49.300859 (+ 587us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:38:49.300874 (+ 15us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:38:49.301186 (+ 312us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:38:49.301437 (+ 251us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:38:49.302321 (+ 884us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:38:49.303411 (+ 1090us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:38:49.304429 (+ 1018us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:38:49.304882 (+ 453us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:38:49.305072 (+ 190us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:38:49.307548 (+ 2476us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:38:49.307557 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:38:49.307568 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 18:38:49.307600 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:38:49.309207 (+ 1607us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:38:49.309710 (+ 503us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:38:49.309717 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:38:49.309725 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:38:49.309873 (+ 148us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:38:49.310185 (+ 312us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:38:49.310189 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:38:49.310191 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:38:49.310535 (+ 344us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:38:49.310745 (+ 210us) server_negotiation.cc:1035] Waiting for connection context
0806 18:38:49.311017 (+ 272us) server_negotiation.cc:299] Negotiation successful
0806 18:38:49.311319 (+ 302us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":37,"server-negotiator.queue_time_us":252,"thread_start_us":122,"threads_started":1}
I20260806 18:38:49.317045 11769 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:38:49.302943 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.194.62:43495 (local address 127.0.0.1:37238)
0806 18:38:49.303378 (+ 435us) negotiation.cc:107] Waiting for socket to connect
0806 18:38:49.303423 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0806 18:38:49.303578 (+ 155us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:38:49.303920 (+ 342us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:38:49.303924 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:38:49.304223 (+ 299us) client_negotiation.cc:189] Negotiated authn=SASL
0806 18:38:49.304331 (+ 108us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:38:49.304336 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:38:49.305081 (+ 745us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:38:49.305084 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:38:49.305697 (+ 613us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:38:49.305705 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:38:49.306233 (+ 528us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:38:49.306237 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:38:49.306356 (+ 119us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:38:49.306881 (+ 525us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 18:38:49.306904 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0806 18:38:49.308483 (+ 1579us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 18:38:49.314619 (+ 6136us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:38:49.314629 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:38:49.314634 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:38:49.315204 (+ 570us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:38:49.315597 (+ 393us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:38:49.315600 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:38:49.315602 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:38:49.315695 (+ 93us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:38:49.316387 (+ 692us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 18:38:49.316397 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 18:38:49.316532 (+ 135us) client_negotiation.cc:769] Sending connection context
0806 18:38:49.316677 (+ 145us) client_negotiation.cc:240] Negotiation successful
0806 18:38:49.316862 (+ 185us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":291,"mutex_wait_us":1,"thread_start_us":178,"threads_started":1}
I20260806 18:38:49.317291 11610 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:38:49.303090 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37238 (local address 127.19.194.62:43495)
0806 18:38:49.303234 (+ 144us) server_negotiation.cc:206] Beginning negotiation
0806 18:38:49.303239 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 18:38:49.303497 (+ 258us) server_negotiation.cc:407] Connection header received
0806 18:38:49.303688 (+ 191us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:38:49.303709 (+ 21us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:38:49.303766 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:38:49.303863 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:38:49.304471 (+ 608us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:38:49.304948 (+ 477us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:38:49.305852 (+ 904us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:38:49.306083 (+ 231us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:38:49.306195 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:38:49.308649 (+ 2454us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:38:49.308652 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:38:49.308654 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 18:38:49.308680 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:38:49.314477 (+ 5797us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:38:49.315346 (+ 869us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:38:49.315349 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:38:49.315351 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:38:49.315456 (+ 105us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:38:49.315810 (+ 354us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:38:49.315813 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:38:49.315815 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:38:49.316105 (+ 290us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:38:49.316209 (+ 104us) server_negotiation.cc:1035] Waiting for connection context
0806 18:38:49.316846 (+ 637us) server_negotiation.cc:299] Negotiation successful
0806 18:38:49.317068 (+ 222us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":17,"server-negotiator.queue_time_us":61}
I20260806 18:38:49.318545 11765 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.19.194.61" port: 36371 } has no permanent_uuid. Determining permanent_uuid...
Aug 06 18:38:49 dist-test-slave-1xrh krb5kdc[11520](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041528, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.19.194.62@KRBTEST.COM
Aug 06 18:38:49 dist-test-slave-1xrh krb5kdc[11520](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041529, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.194.60@KRBTEST.COM for kudu/127.19.194.61@KRBTEST.COM
I20260806 18:38:49.325481 11770 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:38:49.314792 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37252 (local address 127.19.194.62:43495)
0806 18:38:49.315501 (+ 709us) server_negotiation.cc:206] Beginning negotiation
0806 18:38:49.315509 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 18:38:49.315550 (+ 41us) server_negotiation.cc:407] Connection header received
0806 18:38:49.315586 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:38:49.315589 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:38:49.315728 (+ 139us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:38:49.315839 (+ 111us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:38:49.317079 (+ 1240us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:38:49.317848 (+ 769us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:38:49.318967 (+ 1119us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:38:49.319322 (+ 355us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:38:49.319447 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:38:49.321902 (+ 2455us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:38:49.321906 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:38:49.321908 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 18:38:49.321942 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:38:49.323567 (+ 1625us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:38:49.324023 (+ 456us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:38:49.324027 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:38:49.324029 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:38:49.324118 (+ 89us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:38:49.324473 (+ 355us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:38:49.324479 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:38:49.324482 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:38:49.324717 (+ 235us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:38:49.324935 (+ 218us) server_negotiation.cc:1035] Waiting for connection context
0806 18:38:49.325149 (+ 214us) server_negotiation.cc:299] Negotiation successful
0806 18:38:49.325276 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":3,"server-negotiator.queue_time_us":609,"thread_start_us":100,"threads_started":1}
I20260806 18:38:49.327437 11689 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:38:49.319181 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52216 (local address 127.19.194.61:36371)
0806 18:38:49.319315 (+ 134us) server_negotiation.cc:206] Beginning negotiation
0806 18:38:49.319321 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 18:38:49.319338 (+ 17us) server_negotiation.cc:407] Connection header received
0806 18:38:49.319456 (+ 118us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:38:49.319458 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:38:49.319511 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:38:49.319598 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:38:49.319978 (+ 380us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:38:49.320481 (+ 503us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:38:49.321533 (+ 1052us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:38:49.321816 (+ 283us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:38:49.321923 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:38:49.324075 (+ 2152us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:38:49.324079 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:38:49.324082 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 18:38:49.324106 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:38:49.325735 (+ 1629us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:38:49.326147 (+ 412us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:38:49.326151 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:38:49.326153 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:38:49.326247 (+ 94us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:38:49.326570 (+ 323us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:38:49.326573 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:38:49.326575 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:38:49.326803 (+ 228us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:38:49.326904 (+ 101us) server_negotiation.cc:1035] Waiting for connection context
0806 18:38:49.327235 (+ 331us) server_negotiation.cc:299] Negotiation successful
0806 18:38:49.327329 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260806 18:38:49.327533 11769 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:38:49.319171 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.194.61:36371 (local address 127.0.0.1:52216)
0806 18:38:49.319275 (+ 104us) negotiation.cc:107] Waiting for socket to connect
0806 18:38:49.319289 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0806 18:38:49.319366 (+ 77us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:38:49.319610 (+ 244us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:38:49.319613 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:38:49.319776 (+ 163us) client_negotiation.cc:189] Negotiated authn=SASL
0806 18:38:49.319856 (+ 80us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:38:49.319861 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:38:49.320590 (+ 729us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:38:49.320595 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:38:49.321430 (+ 835us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:38:49.321442 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:38:49.321928 (+ 486us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:38:49.321930 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:38:49.322008 (+ 78us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:38:49.322597 (+ 589us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 18:38:49.322618 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0806 18:38:49.323942 (+ 1324us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 18:38:49.325842 (+ 1900us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:38:49.325847 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:38:49.325849 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:38:49.326046 (+ 197us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:38:49.326345 (+ 299us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:38:49.326347 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:38:49.326348 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:38:49.326470 (+ 122us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:38:49.326921 (+ 451us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 18:38:49.326926 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 18:38:49.327020 (+ 94us) client_negotiation.cc:769] Sending connection context
0806 18:38:49.327165 (+ 145us) client_negotiation.cc:240] Negotiation successful
0806 18:38:49.327309 (+ 144us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":30}
I20260806 18:38:49.328197 11765 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.19.194.60" port: 44191 } has no permanent_uuid. Determining permanent_uuid...
Aug 06 18:38:49 dist-test-slave-1xrh krb5kdc[11520](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041528, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.19.194.61@KRBTEST.COM
Aug 06 18:38:49 dist-test-slave-1xrh krb5kdc[11520](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041529, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.194.60@KRBTEST.COM for kudu/127.19.194.60@KRBTEST.COM
I20260806 18:38:49.335886 11688 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:38:49.326787 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52226 (local address 127.19.194.61:36371)
0806 18:38:49.326902 (+ 115us) server_negotiation.cc:206] Beginning negotiation
0806 18:38:49.326906 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 18:38:49.326919 (+ 13us) server_negotiation.cc:407] Connection header received
0806 18:38:49.326950 (+ 31us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:38:49.326952 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:38:49.326990 (+ 38us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:38:49.327052 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:38:49.327955 (+ 903us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:38:49.328572 (+ 617us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:38:49.329566 (+ 994us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:38:49.329814 (+ 248us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:38:49.329891 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:38:49.332627 (+ 2736us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:38:49.332633 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:38:49.332638 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 18:38:49.332666 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:38:49.334202 (+ 1536us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:38:49.334635 (+ 433us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:38:49.334641 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:38:49.334644 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:38:49.334733 (+ 89us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:38:49.334995 (+ 262us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:38:49.335002 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:38:49.335005 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:38:49.335247 (+ 242us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:38:49.335388 (+ 141us) server_negotiation.cc:1035] Waiting for connection context
0806 18:38:49.335629 (+ 241us) server_negotiation.cc:299] Negotiation successful
0806 18:38:49.335758 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260806 18:38:49.336813 11768 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:38:49.328698 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35570 (local address 127.19.194.60:44191)
0806 18:38:49.328802 (+ 104us) server_negotiation.cc:206] Beginning negotiation
0806 18:38:49.328806 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 18:38:49.328862 (+ 56us) server_negotiation.cc:407] Connection header received
0806 18:38:49.328976 (+ 114us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:38:49.328979 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:38:49.329020 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:38:49.329096 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:38:49.329389 (+ 293us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:38:49.329985 (+ 596us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:38:49.330847 (+ 862us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:38:49.331274 (+ 427us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:38:49.331458 (+ 184us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:38:49.333476 (+ 2018us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:38:49.333481 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:38:49.333485 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 18:38:49.333512 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:38:49.335045 (+ 1533us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:38:49.335568 (+ 523us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:38:49.335571 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:38:49.335573 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:38:49.335658 (+ 85us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:38:49.335978 (+ 320us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:38:49.335982 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:38:49.335984 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:38:49.336197 (+ 213us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:38:49.336292 (+ 95us) server_negotiation.cc:1035] Waiting for connection context
0806 18:38:49.336538 (+ 246us) server_negotiation.cc:299] Negotiation successful
0806 18:38:49.336680 (+ 142us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":92,"server-negotiator.queue_time_us":30}
I20260806 18:38:49.336813 11769 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:38:49.328672 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.194.60:44191 (local address 127.0.0.1:35570)
0806 18:38:49.328813 (+ 141us) negotiation.cc:107] Waiting for socket to connect
0806 18:38:49.328824 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0806 18:38:49.328886 (+ 62us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:38:49.329099 (+ 213us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:38:49.329101 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:38:49.329220 (+ 119us) client_negotiation.cc:189] Negotiated authn=SASL
0806 18:38:49.329283 (+ 63us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:38:49.329287 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:38:49.330091 (+ 804us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:38:49.330094 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:38:49.330728 (+ 634us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:38:49.330737 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:38:49.331381 (+ 644us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:38:49.331383 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:38:49.331531 (+ 148us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:38:49.331914 (+ 383us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 18:38:49.331929 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0806 18:38:49.333307 (+ 1378us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 18:38:49.335197 (+ 1890us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:38:49.335200 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:38:49.335202 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:38:49.335445 (+ 243us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:38:49.335769 (+ 324us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:38:49.335772 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:38:49.335777 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:38:49.335872 (+ 95us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:38:49.336292 (+ 420us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 18:38:49.336295 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 18:38:49.336383 (+ 88us) client_negotiation.cc:769] Sending connection context
0806 18:38:49.336495 (+ 112us) client_negotiation.cc:240] Negotiation successful
0806 18:38:49.336626 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":92,"mutex_wait_us":33}
I20260806 18:38:49.339702 20232 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260806 18:38:49.340085 11765 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ee69134877f94c63ade3a98ddeeab9ad: Bootstrap starting.
I20260806 18:38:49.341362 11606 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.19.194.60" port: 44191 } attempt: 3
I20260806 18:38:49.340982 11579 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "00000000000000000000000000000000"
dest_uuid: "a8d621a6deec497b88ee744ab94c452f"
from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:37252
I20260806 18:38:49.342859 11683 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.19.194.60" port: 44191 } attempt: 3
I20260806 18:38:49.342967 11765 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P ee69134877f94c63ade3a98ddeeab9ad: Neither blocks nor log segments found. Creating new log.
I20260806 18:38:49.343890 11765 log.cc:824] T 00000000000000000000000000000000 P ee69134877f94c63ade3a98ddeeab9ad: Log is configured to *not* fsync() on all Append() calls
I20260806 18:38:49.346093 11765 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ee69134877f94c63ade3a98ddeeab9ad: No bootstrap required, opened a new log
Aug 06 18:38:49 dist-test-slave-1xrh krb5kdc[11520](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041528, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.194.62@KRBTEST.COM for kudu/127.19.194.60@KRBTEST.COM
Aug 06 18:38:49 dist-test-slave-1xrh krb5kdc[11520](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041529, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.194.61@KRBTEST.COM for kudu/127.19.194.60@KRBTEST.COM
I20260806 18:38:49.350404 11765 raft_consensus.cc:348] T 00000000000000000000000000000000 P ee69134877f94c63ade3a98ddeeab9ad [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a8d621a6deec497b88ee744ab94c452f" member_type: VOTER last_known_addr { host: "127.19.194.62" port: 43495 } } peers { permanent_uuid: "d874d0e565ce446facb0b9810832d15e" member_type: VOTER last_known_addr { host: "127.19.194.61" port: 36371 } } peers { permanent_uuid: "ee69134877f94c63ade3a98ddeeab9ad" member_type: VOTER last_known_addr { host: "127.19.194.60" port: 44191 } }
I20260806 18:38:49.350668 11765 raft_consensus.cc:729] T 00000000000000000000000000000000 P ee69134877f94c63ade3a98ddeeab9ad [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ee69134877f94c63ade3a98ddeeab9ad, State: Initialized, Role: FOLLOWER
I20260806 18:38:49.350999 11765 consensus_queue.cc:260] T 00000000000000000000000000000000 P ee69134877f94c63ade3a98ddeeab9ad [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a8d621a6deec497b88ee744ab94c452f" member_type: VOTER last_known_addr { host: "127.19.194.62" port: 43495 } } peers { permanent_uuid: "d874d0e565ce446facb0b9810832d15e" member_type: VOTER last_known_addr { host: "127.19.194.61" port: 36371 } } peers { permanent_uuid: "ee69134877f94c63ade3a98ddeeab9ad" member_type: VOTER last_known_addr { host: "127.19.194.60" port: 44191 } }
I20260806 18:38:49.351496 11609 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:38:49.341737 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.194.60:44191 (local address 127.0.0.1:35586)
0806 18:38:49.341883 (+ 146us) negotiation.cc:107] Waiting for socket to connect
0806 18:38:49.341900 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0806 18:38:49.341981 (+ 81us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:38:49.342252 (+ 271us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:38:49.342255 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:38:49.342455 (+ 200us) client_negotiation.cc:189] Negotiated authn=SASL
0806 18:38:49.342550 (+ 95us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:38:49.342555 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:38:49.343573 (+ 1018us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:38:49.343580 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:38:49.344392 (+ 812us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:38:49.344404 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:38:49.344967 (+ 563us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:38:49.344971 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:38:49.345059 (+ 88us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:38:49.345868 (+ 809us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 18:38:49.345883 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0806 18:38:49.347783 (+ 1900us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 18:38:49.349807 (+ 2024us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:38:49.349810 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:38:49.349813 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:38:49.350062 (+ 249us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:38:49.350369 (+ 307us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:38:49.350372 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:38:49.350374 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:38:49.350474 (+ 100us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:38:49.350977 (+ 503us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 18:38:49.350980 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 18:38:49.351078 (+ 98us) client_negotiation.cc:769] Sending connection context
0806 18:38:49.351207 (+ 129us) client_negotiation.cc:240] Negotiation successful
0806 18:38:49.351353 (+ 146us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":71}
I20260806 18:38:49.351529 11773 sys_catalog.cc:455] T 00000000000000000000000000000000 P ee69134877f94c63ade3a98ddeeab9ad [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a8d621a6deec497b88ee744ab94c452f" member_type: VOTER last_known_addr { host: "127.19.194.62" port: 43495 } } peers { permanent_uuid: "d874d0e565ce446facb0b9810832d15e" member_type: VOTER last_known_addr { host: "127.19.194.61" port: 36371 } } peers { permanent_uuid: "ee69134877f94c63ade3a98ddeeab9ad" member_type: VOTER last_known_addr { host: "127.19.194.60" port: 44191 } } }
I20260806 18:38:49.351684 11773 sys_catalog.cc:458] T 00000000000000000000000000000000 P ee69134877f94c63ade3a98ddeeab9ad [sys.catalog]: This master's current role is: FOLLOWER
I20260806 18:38:49.351996 11768 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:38:49.341948 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35586 (local address 127.19.194.60:44191)
0806 18:38:49.342058 (+ 110us) server_negotiation.cc:206] Beginning negotiation
0806 18:38:49.342062 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 18:38:49.342073 (+ 11us) server_negotiation.cc:407] Connection header received
0806 18:38:49.342109 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:38:49.342111 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:38:49.342153 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:38:49.342233 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:38:49.342660 (+ 427us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:38:49.343445 (+ 785us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:38:49.344551 (+ 1106us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:38:49.344841 (+ 290us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:38:49.344966 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:38:49.347945 (+ 2979us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:38:49.347949 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:38:49.347951 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 18:38:49.347976 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:38:49.349682 (+ 1706us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:38:49.350174 (+ 492us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:38:49.350180 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:38:49.350184 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:38:49.350273 (+ 89us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:38:49.350587 (+ 314us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:38:49.350591 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:38:49.350592 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:38:49.350833 (+ 241us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:38:49.350921 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0806 18:38:49.351750 (+ 829us) server_negotiation.cc:299] Negotiation successful
0806 18:38:49.351854 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260806 18:38:49.352411 11765 sys_catalog.cc:564] T 00000000000000000000000000000000 P ee69134877f94c63ade3a98ddeeab9ad [sys.catalog]: configured and running, proceeding with master startup.
I20260806 18:38:49.354538 11686 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:38:49.343290 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.194.60:44191 (local address 127.0.0.1:35596)
0806 18:38:49.343412 (+ 122us) negotiation.cc:107] Waiting for socket to connect
0806 18:38:49.343427 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0806 18:38:49.343507 (+ 80us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:38:49.345442 (+ 1935us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:38:49.345446 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:38:49.345614 (+ 168us) client_negotiation.cc:189] Negotiated authn=SASL
0806 18:38:49.345688 (+ 74us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:38:49.345693 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:38:49.346675 (+ 982us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:38:49.346679 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:38:49.347376 (+ 697us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:38:49.347388 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:38:49.348176 (+ 788us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:38:49.348179 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:38:49.348263 (+ 84us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:38:49.348735 (+ 472us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 18:38:49.348748 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0806 18:38:49.350136 (+ 1388us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 18:38:49.352785 (+ 2649us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:38:49.352791 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:38:49.352794 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:38:49.353037 (+ 243us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:38:49.353442 (+ 405us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:38:49.353447 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:38:49.353448 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:38:49.353538 (+ 90us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:38:49.354000 (+ 462us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 18:38:49.354003 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 18:38:49.354106 (+ 103us) client_negotiation.cc:769] Sending connection context
0806 18:38:49.354232 (+ 126us) client_negotiation.cc:240] Negotiation successful
0806 18:38:49.354370 (+ 138us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":44}
I20260806 18:38:49.355227 11606 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a8d621a6deec497b88ee744ab94c452f: Bootstrap starting.
I20260806 18:38:49.355258 11771 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:38:49.343519 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35596 (local address 127.19.194.60:44191)
0806 18:38:49.345209 (+ 1690us) server_negotiation.cc:206] Beginning negotiation
0806 18:38:49.345221 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0806 18:38:49.345236 (+ 15us) server_negotiation.cc:407] Connection header received
0806 18:38:49.345269 (+ 33us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:38:49.345272 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:38:49.345316 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:38:49.345412 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:38:49.345804 (+ 392us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:38:49.346559 (+ 755us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:38:49.347522 (+ 963us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:38:49.348074 (+ 552us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:38:49.348165 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:38:49.350421 (+ 2256us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:38:49.350424 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:38:49.350426 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 18:38:49.350450 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:38:49.352668 (+ 2218us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:38:49.353179 (+ 511us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:38:49.353185 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:38:49.353187 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:38:49.353311 (+ 124us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:38:49.353682 (+ 371us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:38:49.353684 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:38:49.353686 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:38:49.353881 (+ 195us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:38:49.354866 (+ 985us) server_negotiation.cc:1035] Waiting for connection context
0806 18:38:49.354944 (+ 78us) server_negotiation.cc:299] Negotiation successful
0806 18:38:49.355053 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1610,"thread_start_us":116,"threads_started":1}
I20260806 18:38:49.357808 11606 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P a8d621a6deec497b88ee744ab94c452f: Neither blocks nor log segments found. Creating new log.
I20260806 18:38:49.358567 11683 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d874d0e565ce446facb0b9810832d15e: Bootstrap starting.
I20260806 18:38:49.358662 11606 log.cc:824] T 00000000000000000000000000000000 P a8d621a6deec497b88ee744ab94c452f: Log is configured to *not* fsync() on all Append() calls
I20260806 18:38:49.355877 11690 master_runner.cc:186] Error getting master registration for 127.19.194.62:43495: OK, instance_id { permanent_uuid: "a8d621a6deec497b88ee744ab94c452f" instance_seqno: 1786041528875291 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260806 18:38:49.359185 11690 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
W20260806 18:38:49.359882 11784 catalog_manager.cc:1558] T 00000000000000000000000000000000 P ee69134877f94c63ade3a98ddeeab9ad: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260806 18:38:49.360030 11784 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260806 18:38:49.360770 11606 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a8d621a6deec497b88ee744ab94c452f: No bootstrap required, opened a new log
I20260806 18:38:49.361089 11683 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P d874d0e565ce446facb0b9810832d15e: Neither blocks nor log segments found. Creating new log.
I20260806 18:38:49.361883 11683 log.cc:824] T 00000000000000000000000000000000 P d874d0e565ce446facb0b9810832d15e: Log is configured to *not* fsync() on all Append() calls
I20260806 18:38:49.363840 11606 raft_consensus.cc:348] T 00000000000000000000000000000000 P a8d621a6deec497b88ee744ab94c452f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a8d621a6deec497b88ee744ab94c452f" member_type: VOTER last_known_addr { host: "127.19.194.62" port: 43495 } } peers { permanent_uuid: "d874d0e565ce446facb0b9810832d15e" member_type: VOTER last_known_addr { host: "127.19.194.61" port: 36371 } } peers { permanent_uuid: "ee69134877f94c63ade3a98ddeeab9ad" member_type: VOTER last_known_addr { host: "127.19.194.60" port: 44191 } }
I20260806 18:38:49.364053 11606 raft_consensus.cc:729] T 00000000000000000000000000000000 P a8d621a6deec497b88ee744ab94c452f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a8d621a6deec497b88ee744ab94c452f, State: Initialized, Role: FOLLOWER
I20260806 18:38:49.364156 11683 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d874d0e565ce446facb0b9810832d15e: No bootstrap required, opened a new log
I20260806 18:38:49.364358 11606 consensus_queue.cc:260] T 00000000000000000000000000000000 P a8d621a6deec497b88ee744ab94c452f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a8d621a6deec497b88ee744ab94c452f" member_type: VOTER last_known_addr { host: "127.19.194.62" port: 43495 } } peers { permanent_uuid: "d874d0e565ce446facb0b9810832d15e" member_type: VOTER last_known_addr { host: "127.19.194.61" port: 36371 } } peers { permanent_uuid: "ee69134877f94c63ade3a98ddeeab9ad" member_type: VOTER last_known_addr { host: "127.19.194.60" port: 44191 } }
I20260806 18:38:49.364943 11788 sys_catalog.cc:455] T 00000000000000000000000000000000 P a8d621a6deec497b88ee744ab94c452f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a8d621a6deec497b88ee744ab94c452f" member_type: VOTER last_known_addr { host: "127.19.194.62" port: 43495 } } peers { permanent_uuid: "d874d0e565ce446facb0b9810832d15e" member_type: VOTER last_known_addr { host: "127.19.194.61" port: 36371 } } peers { permanent_uuid: "ee69134877f94c63ade3a98ddeeab9ad" member_type: VOTER last_known_addr { host: "127.19.194.60" port: 44191 } } }
I20260806 18:38:49.365135 11788 sys_catalog.cc:458] T 00000000000000000000000000000000 P a8d621a6deec497b88ee744ab94c452f [sys.catalog]: This master's current role is: FOLLOWER
I20260806 18:38:49.365640 11606 sys_catalog.cc:564] T 00000000000000000000000000000000 P a8d621a6deec497b88ee744ab94c452f [sys.catalog]: configured and running, proceeding with master startup.
I20260806 18:38:49.365677 11579 raft_consensus.cc:482] T 00000000000000000000000000000000 P a8d621a6deec497b88ee744ab94c452f [term 0 FOLLOWER]: Starting forced leader election (received explicit request)
I20260806 18:38:49.365773 11579 raft_consensus.cc:3037] T 00000000000000000000000000000000 P a8d621a6deec497b88ee744ab94c452f [term 0 FOLLOWER]: Advancing to term 1
I20260806 18:38:49.367159 11579 raft_consensus.cc:504] T 00000000000000000000000000000000 P a8d621a6deec497b88ee744ab94c452f [term 1 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a8d621a6deec497b88ee744ab94c452f" member_type: VOTER last_known_addr { host: "127.19.194.62" port: 43495 } } peers { permanent_uuid: "d874d0e565ce446facb0b9810832d15e" member_type: VOTER last_known_addr { host: "127.19.194.61" port: 36371 } } peers { permanent_uuid: "ee69134877f94c63ade3a98ddeeab9ad" member_type: VOTER last_known_addr { host: "127.19.194.60" port: 44191 } }
I20260806 18:38:49.368448 11683 raft_consensus.cc:348] T 00000000000000000000000000000000 P d874d0e565ce446facb0b9810832d15e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a8d621a6deec497b88ee744ab94c452f" member_type: VOTER last_known_addr { host: "127.19.194.62" port: 43495 } } peers { permanent_uuid: "d874d0e565ce446facb0b9810832d15e" member_type: VOTER last_known_addr { host: "127.19.194.61" port: 36371 } } peers { permanent_uuid: "ee69134877f94c63ade3a98ddeeab9ad" member_type: VOTER last_known_addr { host: "127.19.194.60" port: 44191 } }
I20260806 18:38:49.368670 11683 raft_consensus.cc:729] T 00000000000000000000000000000000 P d874d0e565ce446facb0b9810832d15e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d874d0e565ce446facb0b9810832d15e, State: Initialized, Role: FOLLOWER
I20260806 18:38:49.368985 11683 consensus_queue.cc:260] T 00000000000000000000000000000000 P d874d0e565ce446facb0b9810832d15e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a8d621a6deec497b88ee744ab94c452f" member_type: VOTER last_known_addr { host: "127.19.194.62" port: 43495 } } peers { permanent_uuid: "d874d0e565ce446facb0b9810832d15e" member_type: VOTER last_known_addr { host: "127.19.194.61" port: 36371 } } peers { permanent_uuid: "ee69134877f94c63ade3a98ddeeab9ad" member_type: VOTER last_known_addr { host: "127.19.194.60" port: 44191 } }
I20260806 18:38:49.369295 11736 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "a8d621a6deec497b88ee744ab94c452f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "ee69134877f94c63ade3a98ddeeab9ad"
I20260806 18:38:49.369555 11736 raft_consensus.cc:3037] T 00000000000000000000000000000000 P ee69134877f94c63ade3a98ddeeab9ad [term 0 FOLLOWER]: Advancing to term 1
I20260806 18:38:49.370293 11683 sys_catalog.cc:564] T 00000000000000000000000000000000 P d874d0e565ce446facb0b9810832d15e [sys.catalog]: configured and running, proceeding with master startup.
I20260806 18:38:49.371222 11655 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "a8d621a6deec497b88ee744ab94c452f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "d874d0e565ce446facb0b9810832d15e"
I20260806 18:38:49.371423 11655 raft_consensus.cc:3037] T 00000000000000000000000000000000 P d874d0e565ce446facb0b9810832d15e [term 0 FOLLOWER]: Advancing to term 1
I20260806 18:38:49.372732 11655 raft_consensus.cc:2454] T 00000000000000000000000000000000 P d874d0e565ce446facb0b9810832d15e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a8d621a6deec497b88ee744ab94c452f in term 1.
W20260806 18:38:49.373061 11556 tablet.cc:2369] T 00000000000000000000000000000000 P a8d621a6deec497b88ee744ab94c452f: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 18:38:49.373198 11554 leader_election.cc:302] T 00000000000000000000000000000000 P a8d621a6deec497b88ee744ab94c452f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a8d621a6deec497b88ee744ab94c452f, d874d0e565ce446facb0b9810832d15e; no voters:
I20260806 18:38:49.373553 11788 raft_consensus.cc:2781] T 00000000000000000000000000000000 P a8d621a6deec497b88ee744ab94c452f [term 1 FOLLOWER]: Leader election won for term 1
I20260806 18:38:49.373806 11788 raft_consensus.cc:686] T 00000000000000000000000000000000 P a8d621a6deec497b88ee744ab94c452f [term 1 LEADER]: Becoming Leader. State: Replica: a8d621a6deec497b88ee744ab94c452f, State: Running, Role: LEADER
I20260806 18:38:49.373865 11736 raft_consensus.cc:2454] T 00000000000000000000000000000000 P ee69134877f94c63ade3a98ddeeab9ad [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a8d621a6deec497b88ee744ab94c452f in term 1.
I20260806 18:38:49.368163 11579 leader_election.cc:288] T 00000000000000000000000000000000 P a8d621a6deec497b88ee744ab94c452f [CANDIDATE]: Term 1 election: Requested vote from peers d874d0e565ce446facb0b9810832d15e (127.19.194.61:36371), ee69134877f94c63ade3a98ddeeab9ad (127.19.194.60:44191)
I20260806 18:38:49.374193 11788 consensus_queue.cc:237] T 00000000000000000000000000000000 P a8d621a6deec497b88ee744ab94c452f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a8d621a6deec497b88ee744ab94c452f" member_type: VOTER last_known_addr { host: "127.19.194.62" port: 43495 } } peers { permanent_uuid: "d874d0e565ce446facb0b9810832d15e" member_type: VOTER last_known_addr { host: "127.19.194.61" port: 36371 } } peers { permanent_uuid: "ee69134877f94c63ade3a98ddeeab9ad" member_type: VOTER last_known_addr { host: "127.19.194.60" port: 44191 } }
I20260806 18:38:49.371207 11535 master_runner.cc:186] Error getting master registration for 127.19.194.61:36371: OK, instance_id { permanent_uuid: "d874d0e565ce446facb0b9810832d15e" instance_seqno: 1786041529086046 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260806 18:38:49.374617 11535 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
W20260806 18:38:49.378922 11813 catalog_manager.cc:1558] T 00000000000000000000000000000000 P d874d0e565ce446facb0b9810832d15e: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260806 18:38:49.379000 11813 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260806 18:38:49.380059 11788 sys_catalog.cc:455] T 00000000000000000000000000000000 P a8d621a6deec497b88ee744ab94c452f [sys.catalog]: SysCatalogTable state changed. Reason: New leader a8d621a6deec497b88ee744ab94c452f. Latest consensus state: current_term: 1 leader_uuid: "a8d621a6deec497b88ee744ab94c452f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a8d621a6deec497b88ee744ab94c452f" member_type: VOTER last_known_addr { host: "127.19.194.62" port: 43495 } } peers { permanent_uuid: "d874d0e565ce446facb0b9810832d15e" member_type: VOTER last_known_addr { host: "127.19.194.61" port: 36371 } } peers { permanent_uuid: "ee69134877f94c63ade3a98ddeeab9ad" member_type: VOTER last_known_addr { host: "127.19.194.60" port: 44191 } } }
I20260806 18:38:49.380215 11788 sys_catalog.cc:458] T 00000000000000000000000000000000 P a8d621a6deec497b88ee744ab94c452f [sys.catalog]: This master's current role is: LEADER
W20260806 18:38:49.381268 11799 catalog_manager.cc:1558] T 00000000000000000000000000000000 P a8d621a6deec497b88ee744ab94c452f: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260806 18:38:49.381343 11799 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260806 18:38:49.383095 11800 sys_catalog.cc:455] T 00000000000000000000000000000000 P d874d0e565ce446facb0b9810832d15e [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a8d621a6deec497b88ee744ab94c452f" member_type: VOTER last_known_addr { host: "127.19.194.62" port: 43495 } } peers { permanent_uuid: "d874d0e565ce446facb0b9810832d15e" member_type: VOTER last_known_addr { host: "127.19.194.61" port: 36371 } } peers { permanent_uuid: "ee69134877f94c63ade3a98ddeeab9ad" member_type: VOTER last_known_addr { host: "127.19.194.60" port: 44191 } } }
I20260806 18:38:49.383271 11800 sys_catalog.cc:458] T 00000000000000000000000000000000 P d874d0e565ce446facb0b9810832d15e [sys.catalog]: This master's current role is: FOLLOWER
I20260806 18:38:49.383487 11822 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 18:38:49.384402 11822 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 18:38:49.395255 11771 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:38:49.376591 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34488 (local address 127.19.194.60:44191)
0806 18:38:49.376712 (+ 121us) server_negotiation.cc:206] Beginning negotiation
0806 18:38:49.376717 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 18:38:49.384752 (+ 8035us) server_negotiation.cc:407] Connection header received
0806 18:38:49.384854 (+ 102us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:38:49.384857 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:38:49.384927 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:38:49.385008 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:38:49.386142 (+ 1134us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:38:49.387053 (+ 911us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:38:49.388437 (+ 1384us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:38:49.388858 (+ 421us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:38:49.388940 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:38:49.390991 (+ 2051us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:38:49.390996 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:38:49.390999 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 18:38:49.391027 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:38:49.393054 (+ 2027us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:38:49.393594 (+ 540us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:38:49.393598 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:38:49.393600 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:38:49.393708 (+ 108us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:38:49.394064 (+ 356us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:38:49.394068 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:38:49.394070 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:38:49.394316 (+ 246us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:38:49.394457 (+ 141us) server_negotiation.cc:1035] Waiting for connection context
0806 18:38:49.394892 (+ 435us) server_negotiation.cc:299] Negotiation successful
0806 18:38:49.395031 (+ 139us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260806 18:38:49.406728 11770 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:38:49.381627 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58194 (local address 127.19.194.62:43495)
0806 18:38:49.381723 (+ 96us) server_negotiation.cc:206] Beginning negotiation
0806 18:38:49.381727 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 18:38:49.395150 (+ 13423us) server_negotiation.cc:407] Connection header received
0806 18:38:49.395340 (+ 190us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:38:49.395356 (+ 16us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:38:49.395413 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:38:49.395474 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:38:49.397447 (+ 1973us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:38:49.398367 (+ 920us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:38:49.399675 (+ 1308us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:38:49.400177 (+ 502us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:38:49.400264 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:38:49.402007 (+ 1743us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:38:49.402011 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:38:49.402014 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 18:38:49.402041 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:38:49.403957 (+ 1916us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:38:49.404504 (+ 547us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:38:49.404508 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:38:49.404510 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:38:49.404602 (+ 92us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:38:49.405284 (+ 682us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:38:49.405288 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:38:49.405300 (+ 12us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:38:49.405514 (+ 214us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:38:49.405635 (+ 121us) server_negotiation.cc:1035] Waiting for connection context
0806 18:38:49.406443 (+ 808us) server_negotiation.cc:299] Negotiation successful
0806 18:38:49.406568 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260806 18:38:49.409271 11688 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:38:49.380851 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52116 (local address 127.19.194.61:36371)
0806 18:38:49.381496 (+ 645us) server_negotiation.cc:206] Beginning negotiation
0806 18:38:49.381501 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 18:38:49.391427 (+ 9926us) server_negotiation.cc:407] Connection header received
0806 18:38:49.391648 (+ 221us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:38:49.391651 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:38:49.391708 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:38:49.391771 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:38:49.392850 (+ 1079us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:38:49.393744 (+ 894us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:38:49.396623 (+ 2879us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:38:49.396935 (+ 312us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:38:49.397000 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:38:49.398699 (+ 1699us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:38:49.398703 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:38:49.398706 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 18:38:49.398734 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:38:49.400611 (+ 1877us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:38:49.405969 (+ 5358us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:38:49.405973 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:38:49.405975 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:38:49.406065 (+ 90us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:38:49.408172 (+ 2107us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:38:49.408176 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:38:49.408177 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:38:49.408422 (+ 245us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:38:49.408543 (+ 121us) server_negotiation.cc:1035] Waiting for connection context
0806 18:38:49.408994 (+ 451us) server_negotiation.cc:299] Negotiation successful
0806 18:38:49.409120 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":561}
I20260806 18:38:49.411237 11736 raft_consensus.cc:1261] T 00000000000000000000000000000000 P ee69134877f94c63ade3a98ddeeab9ad [term 1 FOLLOWER]: Refusing update from remote peer a8d621a6deec497b88ee744ab94c452f: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 18:38:49.412156 11788 consensus_queue.cc:1035] T 00000000000000000000000000000000 P a8d621a6deec497b88ee744ab94c452f [LEADER]: Connected to new peer: Peer: permanent_uuid: "ee69134877f94c63ade3a98ddeeab9ad" member_type: VOTER last_known_addr { host: "127.19.194.60" port: 44191 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 18:38:49.413211 11655 raft_consensus.cc:1261] T 00000000000000000000000000000000 P d874d0e565ce446facb0b9810832d15e [term 1 FOLLOWER]: Refusing update from remote peer a8d621a6deec497b88ee744ab94c452f: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 18:38:49.414278 11812 consensus_queue.cc:1035] T 00000000000000000000000000000000 P a8d621a6deec497b88ee744ab94c452f [LEADER]: Connected to new peer: Peer: permanent_uuid: "d874d0e565ce446facb0b9810832d15e" member_type: VOTER last_known_addr { host: "127.19.194.61" port: 36371 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 18:38:49.419099 11800 sys_catalog.cc:455] T 00000000000000000000000000000000 P d874d0e565ce446facb0b9810832d15e [sys.catalog]: SysCatalogTable state changed. Reason: New leader a8d621a6deec497b88ee744ab94c452f. Latest consensus state: current_term: 1 leader_uuid: "a8d621a6deec497b88ee744ab94c452f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a8d621a6deec497b88ee744ab94c452f" member_type: VOTER last_known_addr { host: "127.19.194.62" port: 43495 } } peers { permanent_uuid: "d874d0e565ce446facb0b9810832d15e" member_type: VOTER last_known_addr { host: "127.19.194.61" port: 36371 } } peers { permanent_uuid: "ee69134877f94c63ade3a98ddeeab9ad" member_type: VOTER last_known_addr { host: "127.19.194.60" port: 44191 } } }
I20260806 18:38:49.419265 11800 sys_catalog.cc:458] T 00000000000000000000000000000000 P d874d0e565ce446facb0b9810832d15e [sys.catalog]: This master's current role is: FOLLOWER
I20260806 18:38:49.423743 11773 sys_catalog.cc:455] T 00000000000000000000000000000000 P ee69134877f94c63ade3a98ddeeab9ad [sys.catalog]: SysCatalogTable state changed. Reason: New leader a8d621a6deec497b88ee744ab94c452f. Latest consensus state: current_term: 1 leader_uuid: "a8d621a6deec497b88ee744ab94c452f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a8d621a6deec497b88ee744ab94c452f" member_type: VOTER last_known_addr { host: "127.19.194.62" port: 43495 } } peers { permanent_uuid: "d874d0e565ce446facb0b9810832d15e" member_type: VOTER last_known_addr { host: "127.19.194.61" port: 36371 } } peers { permanent_uuid: "ee69134877f94c63ade3a98ddeeab9ad" member_type: VOTER last_known_addr { host: "127.19.194.60" port: 44191 } } }
I20260806 18:38:49.423892 11773 sys_catalog.cc:458] T 00000000000000000000000000000000 P ee69134877f94c63ade3a98ddeeab9ad [sys.catalog]: This master's current role is: FOLLOWER
I20260806 18:38:49.431355 11788 sys_catalog.cc:455] T 00000000000000000000000000000000 P a8d621a6deec497b88ee744ab94c452f [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "a8d621a6deec497b88ee744ab94c452f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a8d621a6deec497b88ee744ab94c452f" member_type: VOTER last_known_addr { host: "127.19.194.62" port: 43495 } } peers { permanent_uuid: "d874d0e565ce446facb0b9810832d15e" member_type: VOTER last_known_addr { host: "127.19.194.61" port: 36371 } } peers { permanent_uuid: "ee69134877f94c63ade3a98ddeeab9ad" member_type: VOTER last_known_addr { host: "127.19.194.60" port: 44191 } } }
I20260806 18:38:49.431488 11788 sys_catalog.cc:458] T 00000000000000000000000000000000 P a8d621a6deec497b88ee744ab94c452f [sys.catalog]: This master's current role is: LEADER
I20260806 18:38:49.431625 11825 mvcc.cc:205] Tried to move back new op lower bound from 7315626104458924032 to 7315626104328949760. Current Snapshot: MvccSnapshot[applied={T|T < 7315626104458924032}]
I20260806 18:38:49.435340 11800 sys_catalog.cc:455] T 00000000000000000000000000000000 P d874d0e565ce446facb0b9810832d15e [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "a8d621a6deec497b88ee744ab94c452f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a8d621a6deec497b88ee744ab94c452f" member_type: VOTER last_known_addr { host: "127.19.194.62" port: 43495 } } peers { permanent_uuid: "d874d0e565ce446facb0b9810832d15e" member_type: VOTER last_known_addr { host: "127.19.194.61" port: 36371 } } peers { permanent_uuid: "ee69134877f94c63ade3a98ddeeab9ad" member_type: VOTER last_known_addr { host: "127.19.194.60" port: 44191 } } }
I20260806 18:38:49.435462 11800 sys_catalog.cc:458] T 00000000000000000000000000000000 P d874d0e565ce446facb0b9810832d15e [sys.catalog]: This master's current role is: FOLLOWER
I20260806 18:38:49.436513 11822 catalog_manager.cc:1347] Generated new cluster ID: 0281838d271f4d63b019bb9439cece9f
I20260806 18:38:49.436576 11822 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 18:38:49.438283 11812 sys_catalog.cc:455] T 00000000000000000000000000000000 P a8d621a6deec497b88ee744ab94c452f [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "a8d621a6deec497b88ee744ab94c452f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a8d621a6deec497b88ee744ab94c452f" member_type: VOTER last_known_addr { host: "127.19.194.62" port: 43495 } } peers { permanent_uuid: "d874d0e565ce446facb0b9810832d15e" member_type: VOTER last_known_addr { host: "127.19.194.61" port: 36371 } } peers { permanent_uuid: "ee69134877f94c63ade3a98ddeeab9ad" member_type: VOTER last_known_addr { host: "127.19.194.60" port: 44191 } } }
I20260806 18:38:49.438398 11812 sys_catalog.cc:458] T 00000000000000000000000000000000 P a8d621a6deec497b88ee744ab94c452f [sys.catalog]: This master's current role is: LEADER
I20260806 18:38:49.441846 11773 sys_catalog.cc:455] T 00000000000000000000000000000000 P ee69134877f94c63ade3a98ddeeab9ad [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "a8d621a6deec497b88ee744ab94c452f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a8d621a6deec497b88ee744ab94c452f" member_type: VOTER last_known_addr { host: "127.19.194.62" port: 43495 } } peers { permanent_uuid: "d874d0e565ce446facb0b9810832d15e" member_type: VOTER last_known_addr { host: "127.19.194.61" port: 36371 } } peers { permanent_uuid: "ee69134877f94c63ade3a98ddeeab9ad" member_type: VOTER last_known_addr { host: "127.19.194.60" port: 44191 } } }
I20260806 18:38:49.441949 11773 sys_catalog.cc:458] T 00000000000000000000000000000000 P ee69134877f94c63ade3a98ddeeab9ad [sys.catalog]: This master's current role is: FOLLOWER
I20260806 18:38:49.442413 11828 mvcc.cc:205] Tried to move back new op lower bound from 7315626104458924032 to 7315626104328949760. Current Snapshot: MvccSnapshot[applied={T|T < 7315626104458924032}]
I20260806 18:38:49.464380 11822 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 18:38:49.465689 11822 catalog_manager.cc:1504] Loading token signing keys...
I20260806 18:38:49.478917 11822 catalog_manager.cc:5868] T 00000000000000000000000000000000 P a8d621a6deec497b88ee744ab94c452f: Generated new TSK 0
I20260806 18:38:49.479745 11822 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260806 18:38:49.509244 11688 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 18:38:49.506586 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52124 (local address 127.19.194.61:36371)
0806 18:38:49.506716 (+ 130us) server_negotiation.cc:206] Beginning negotiation
0806 18:38:49.506722 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 18:38:49.508121 (+ 1399us) server_negotiation.cc:407] Connection header received
0806 18:38:49.508342 (+ 221us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:38:49.508346 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:38:49.508399 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:38:49.508454 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:38:49.509107 (+ 653us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:52124: BlockingRecv error: recv got EOF from 127.0.0.1:52124 (error 108)
Metrics: {"server-negotiator.queue_time_us":50}
W20260806 18:38:49.510816 11771 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 18:38:49.507443 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34502 (local address 127.19.194.60:44191)
0806 18:38:49.507563 (+ 120us) server_negotiation.cc:206] Beginning negotiation
0806 18:38:49.507568 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 18:38:49.509796 (+ 2228us) server_negotiation.cc:407] Connection header received
0806 18:38:49.510032 (+ 236us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:38:49.510036 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:38:49.510085 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:38:49.510136 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:38:49.510676 (+ 540us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:34502: BlockingRecv error: recv got EOF from 127.0.0.1:34502 (error 108)
Metrics: {"server-negotiator.queue_time_us":44}
W20260806 18:38:49.512894 11770 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 18:38:49.511354 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58206 (local address 127.19.194.62:43495)
0806 18:38:49.511468 (+ 114us) server_negotiation.cc:206] Beginning negotiation
0806 18:38:49.511472 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 18:38:49.511948 (+ 476us) server_negotiation.cc:407] Connection header received
0806 18:38:49.512130 (+ 182us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:38:49.512133 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:38:49.512184 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:38:49.512238 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:38:49.512774 (+ 536us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:58206: BlockingRecv error: recv got EOF from 127.0.0.1:58206 (error 108)
Metrics: {"server-negotiator.queue_time_us":46}
W20260806 18:38:49.517140 11688 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 18:38:49.515601 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52138 (local address 127.19.194.61:36371)
0806 18:38:49.515703 (+ 102us) server_negotiation.cc:206] Beginning negotiation
0806 18:38:49.515708 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 18:38:49.516204 (+ 496us) server_negotiation.cc:407] Connection header received
0806 18:38:49.516420 (+ 216us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:38:49.516425 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:38:49.516495 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:38:49.516545 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:38:49.517031 (+ 486us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:52138: BlockingRecv error: recv got EOF from 127.0.0.1:52138 (error 108)
Metrics: {"server-negotiator.queue_time_us":36}
W20260806 18:38:49.521739 11688 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 18:38:49.520396 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52150 (local address 127.19.194.61:36371)
0806 18:38:49.520487 (+ 91us) server_negotiation.cc:206] Beginning negotiation
0806 18:38:49.520491 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 18:38:49.520966 (+ 475us) server_negotiation.cc:407] Connection header received
0806 18:38:49.521127 (+ 161us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:38:49.521130 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:38:49.521174 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:38:49.521219 (+ 45us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:38:49.521654 (+ 435us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:52150: BlockingRecv error: recv got EOF from 127.0.0.1:52150 (error 108)
Metrics: {"server-negotiator.queue_time_us":35}
W20260806 18:38:49.527091 11688 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 18:38:49.525593 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52162 (local address 127.19.194.61:36371)
0806 18:38:49.525692 (+ 99us) server_negotiation.cc:206] Beginning negotiation
0806 18:38:49.525696 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 18:38:49.526221 (+ 525us) server_negotiation.cc:407] Connection header received
0806 18:38:49.526423 (+ 202us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:38:49.526426 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:38:49.526473 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:38:49.526524 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:38:49.527005 (+ 481us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:52162: BlockingRecv error: recv got EOF from 127.0.0.1:52162 (error 108)
Metrics: {"server-negotiator.queue_time_us":40}
W20260806 18:38:49.534529 11688 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 18:38:49.532974 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52174 (local address 127.19.194.61:36371)
0806 18:38:49.533096 (+ 122us) server_negotiation.cc:206] Beginning negotiation
0806 18:38:49.533100 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 18:38:49.533626 (+ 526us) server_negotiation.cc:407] Connection header received
0806 18:38:49.533831 (+ 205us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:38:49.533834 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:38:49.533883 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:38:49.533934 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:38:49.534437 (+ 503us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:52174: BlockingRecv error: recv got EOF from 127.0.0.1:52174 (error 108)
Metrics: {"server-negotiator.queue_time_us":63}
W20260806 18:38:49.545972 11688 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 18:38:49.544478 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52188 (local address 127.19.194.61:36371)
0806 18:38:49.544577 (+ 99us) server_negotiation.cc:206] Beginning negotiation
0806 18:38:49.544581 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 18:38:49.545113 (+ 532us) server_negotiation.cc:407] Connection header received
0806 18:38:49.545306 (+ 193us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:38:49.545309 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:38:49.545356 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:38:49.545405 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:38:49.545880 (+ 475us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:52188: BlockingRecv error: recv got EOF from 127.0.0.1:52188 (error 108)
Metrics: {"server-negotiator.queue_time_us":40}
W20260806 18:38:49.566192 11688 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 18:38:49.564471 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52206 (local address 127.19.194.61:36371)
0806 18:38:49.564585 (+ 114us) server_negotiation.cc:206] Beginning negotiation
0806 18:38:49.564590 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 18:38:49.565228 (+ 638us) server_negotiation.cc:407] Connection header received
0806 18:38:49.565452 (+ 224us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:38:49.565456 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:38:49.565507 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:38:49.565561 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:38:49.566088 (+ 527us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:52206: BlockingRecv error: recv got EOF from 127.0.0.1:52206 (error 108)
Metrics: {"server-negotiator.queue_time_us":44}
W20260806 18:38:49.602103 11688 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 18:38:49.600876 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52220 (local address 127.19.194.61:36371)
0806 18:38:49.601004 (+ 128us) server_negotiation.cc:206] Beginning negotiation
0806 18:38:49.601009 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 18:38:49.601217 (+ 208us) server_negotiation.cc:407] Connection header received
0806 18:38:49.601347 (+ 130us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:38:49.601350 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:38:49.601405 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:38:49.601501 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:38:49.601982 (+ 481us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:52220: BlockingRecv error: recv got EOF from 127.0.0.1:52220 (error 108)
Metrics: {"server-negotiator.queue_time_us":35}
W20260806 18:38:49.669397 11688 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 18:38:49.668396 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52234 (local address 127.19.194.61:36371)
0806 18:38:49.668524 (+ 128us) server_negotiation.cc:206] Beginning negotiation
0806 18:38:49.668528 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 18:38:49.668650 (+ 122us) server_negotiation.cc:407] Connection header received
0806 18:38:49.668758 (+ 108us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:38:49.668761 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:38:49.668814 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:38:49.668896 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:38:49.669288 (+ 392us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:52234: BlockingRecv error: recv got EOF from 127.0.0.1:52234 (error 108)
Metrics: {"server-negotiator.queue_time_us":46}
W20260806 18:38:49.800848 11688 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 18:38:49.799742 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52240 (local address 127.19.194.61:36371)
0806 18:38:49.799897 (+ 155us) server_negotiation.cc:206] Beginning negotiation
0806 18:38:49.799902 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 18:38:49.800048 (+ 146us) server_negotiation.cc:407] Connection header received
0806 18:38:49.800153 (+ 105us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:38:49.800155 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:38:49.800202 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:38:49.800270 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:38:49.800708 (+ 438us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:52240: BlockingRecv error: recv got EOF from 127.0.0.1:52240 (error 108)
Metrics: {"server-negotiator.queue_time_us":62}
W20260806 18:38:50.060115 11688 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 18:38:50.058920 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52254 (local address 127.19.194.61:36371)
0806 18:38:50.059062 (+ 142us) server_negotiation.cc:206] Beginning negotiation
0806 18:38:50.059069 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 18:38:50.059170 (+ 101us) server_negotiation.cc:407] Connection header received
0806 18:38:50.059354 (+ 184us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:38:50.059358 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:38:50.059412 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:38:50.059506 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:38:50.059981 (+ 475us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:52254: BlockingRecv error: recv got EOF from 127.0.0.1:52254 (error 108)
Metrics: {"server-negotiator.queue_time_us":57}
I20260806 18:38:50.361585 11784 catalog_manager.cc:1259] Loaded cluster ID: 0281838d271f4d63b019bb9439cece9f
I20260806 18:38:50.361667 11784 catalog_manager.cc:1552] T 00000000000000000000000000000000 P ee69134877f94c63ade3a98ddeeab9ad: loading cluster ID for follower catalog manager: success
I20260806 18:38:50.363549 11784 catalog_manager.cc:1574] T 00000000000000000000000000000000 P ee69134877f94c63ade3a98ddeeab9ad: acquiring CA information for follower catalog manager: success
I20260806 18:38:50.364818 11784 catalog_manager.cc:1602] T 00000000000000000000000000000000 P ee69134877f94c63ade3a98ddeeab9ad: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260806 18:38:50.380295 11813 catalog_manager.cc:1259] Loaded cluster ID: 0281838d271f4d63b019bb9439cece9f
I20260806 18:38:50.380371 11813 catalog_manager.cc:1552] T 00000000000000000000000000000000 P d874d0e565ce446facb0b9810832d15e: loading cluster ID for follower catalog manager: success
I20260806 18:38:50.381913 11813 catalog_manager.cc:1574] T 00000000000000000000000000000000 P d874d0e565ce446facb0b9810832d15e: acquiring CA information for follower catalog manager: success
I20260806 18:38:50.382972 11813 catalog_manager.cc:1602] T 00000000000000000000000000000000 P d874d0e565ce446facb0b9810832d15e: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260806 18:38:50.577764 11899 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:38:50.574137 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52264 (local address 127.19.194.61:36371)
0806 18:38:50.574449 (+ 312us) server_negotiation.cc:206] Beginning negotiation
0806 18:38:50.574452 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 18:38:50.574470 (+ 18us) server_negotiation.cc:407] Connection header received
0806 18:38:50.574546 (+ 76us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:38:50.574548 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:38:50.574677 (+ 129us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:38:50.574775 (+ 98us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 18:38:50.575020 (+ 245us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:38:50.575547 (+ 527us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:38:50.576351 (+ 804us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:38:50.576566 (+ 215us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:38:50.576639 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:38:50.576918 (+ 279us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 18:38:50.577350 (+ 432us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 18:38:50.577445 (+ 95us) server_negotiation.cc:1035] Waiting for connection context
0806 18:38:50.577506 (+ 61us) server_negotiation.cc:299] Negotiation successful
0806 18:38:50.577553 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":232,"thread_start_us":102,"threads_started":1}
I20260806 18:38:50.584230 11906 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:38:50.580669 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34504 (local address 127.19.194.60:44191)
0806 18:38:50.580898 (+ 229us) server_negotiation.cc:206] Beginning negotiation
0806 18:38:50.580902 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 18:38:50.580956 (+ 54us) server_negotiation.cc:407] Connection header received
0806 18:38:50.581112 (+ 156us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:38:50.581114 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:38:50.581221 (+ 107us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:38:50.581311 (+ 90us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 18:38:50.581510 (+ 199us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:38:50.582009 (+ 499us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:38:50.582793 (+ 784us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:38:50.583018 (+ 225us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:38:50.583094 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:38:50.583408 (+ 314us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 18:38:50.583840 (+ 432us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 18:38:50.583938 (+ 98us) server_negotiation.cc:1035] Waiting for connection context
0806 18:38:50.584025 (+ 87us) server_negotiation.cc:299] Negotiation successful
0806 18:38:50.584069 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":159,"thread_start_us":61,"threads_started":1}
I20260806 18:38:50.591310 11906 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:38:50.587125 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34514 (local address 127.19.194.60:44191)
0806 18:38:50.587245 (+ 120us) server_negotiation.cc:206] Beginning negotiation
0806 18:38:50.587249 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 18:38:50.587770 (+ 521us) server_negotiation.cc:407] Connection header received
0806 18:38:50.587818 (+ 48us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:38:50.587821 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:38:50.587873 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:38:50.587952 (+ 79us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 18:38:50.588146 (+ 194us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:38:50.588916 (+ 770us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:38:50.590063 (+ 1147us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:38:50.590392 (+ 329us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:38:50.590485 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:38:50.590854 (+ 369us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 18:38:50.590926 (+ 72us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 18:38:50.591005 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0806 18:38:50.591098 (+ 93us) server_negotiation.cc:299] Negotiation successful
0806 18:38:50.591166 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":29}
I20260806 18:38:50.591817 11912 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:38:50.587161 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58222 (local address 127.19.194.62:43495)
0806 18:38:50.587814 (+ 653us) server_negotiation.cc:206] Beginning negotiation
0806 18:38:50.587818 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 18:38:50.587830 (+ 12us) server_negotiation.cc:407] Connection header received
0806 18:38:50.587916 (+ 86us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:38:50.587921 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:38:50.588068 (+ 147us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:38:50.588206 (+ 138us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 18:38:50.588306 (+ 100us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:38:50.589070 (+ 764us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:38:50.590119 (+ 1049us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:38:50.590347 (+ 228us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:38:50.590459 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:38:50.590756 (+ 297us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 18:38:50.591058 (+ 302us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 18:38:50.591169 (+ 111us) server_negotiation.cc:1035] Waiting for connection context
0806 18:38:50.591594 (+ 425us) server_negotiation.cc:299] Negotiation successful
0806 18:38:50.591664 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":555,"thread_start_us":87,"threads_started":1}
I20260806 18:38:50.592832 11899 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:38:50.587389 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52280 (local address 127.19.194.61:36371)
0806 18:38:50.587506 (+ 117us) server_negotiation.cc:206] Beginning negotiation
0806 18:38:50.587511 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 18:38:50.589337 (+ 1826us) server_negotiation.cc:407] Connection header received
0806 18:38:50.589478 (+ 141us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:38:50.589481 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:38:50.589550 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:38:50.589635 (+ 85us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 18:38:50.589881 (+ 246us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:38:50.590585 (+ 704us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:38:50.591744 (+ 1159us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:38:50.592045 (+ 301us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:38:50.592154 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:38:50.592418 (+ 264us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 18:38:50.592501 (+ 83us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 18:38:50.592567 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0806 18:38:50.592655 (+ 88us) server_negotiation.cc:299] Negotiation successful
0806 18:38:50.592702 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260806 18:38:50.594787 20232 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJW69tn/build/debug/bin/kudu with pid 11535
I20260806 18:38:50.603618 20232 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJW69tn/build/debug/bin/kudu with pid 11613
I20260806 18:38:50.610499 20232 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJW69tn/build/debug/bin/kudu with pid 11690
2026-08-06T18:38:50Z chronyd exiting
[ OK ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/1 (3081 ms)
[----------] 2 tests from ConnectToFollowerMasterTest (6169 ms total)
[----------] Global test environment tear-down
[==========] 56 tests from 4 test suites ran. (284510 ms total)
[ PASSED ] 55 tests.
[ FAILED ] 1 test, listed below:
[ FAILED ] SecurityITest.TestJwtMiniCluster
1 FAILED TEST